{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 机器学习练习-决策树\n",
    "代码更新地址：https://github.com/fengdu78/WZU-machine-learning-course\n",
    "\n",
    "代码修改并注释：黄海广，haiguang2000@wzu.edu.cn "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1．分类决策树模型是表示基于特征对实例进行分类的树形结构。决策树可以转换成一个**if-then**规则的集合，也可以看作是定义在特征空间划分上的类的条件概率分布。\n",
    "\n",
    "2．决策树学习旨在构建一个与训练数据拟合很好，并且复杂度小的决策树。因为从可能的决策树中直接选取最优决策树是NP完全问题。现实中采用启发式方法学习次优的决策树。\n",
    "\n",
    "决策树学习算法包括3部分：特征选择、树的生成和树的剪枝。常用的算法有ID3、\n",
    "C4.5和CART。\n",
    "\n",
    "3．特征选择的目的在于选取对训练数据能够分类的特征。特征选择的关键是其准则。常用的准则如下：\n",
    "\n",
    "（1）样本集合$D$对特征$A$的信息增益（ID3）\n",
    "\n",
    "\n",
    "$$g(D, A)=H(D)-H(D|A)$$\n",
    "\n",
    "$$H(D)=-\\sum_{k=1}^{K} \\frac{\\left|C_{k}\\right|}{|D|} \\log _{2} \\frac{\\left|C_{k}\\right|}{|D|}$$\n",
    "\n",
    "$$H(D | A)=\\sum_{i=1}^{n} \\frac{\\left|D_{i}\\right|}{|D|} H\\left(D_{i}\\right)$$\n",
    "\n",
    "其中，$H(D)$是数据集$D$的熵，$H(D_i)$是数据集$D_i$的熵，$H(D|A)$是数据集$D$对特征$A$的条件熵。\t$D_i$是$D$中特征$A$取第$i$个值的样本子集，$C_k$是$D$中属于第$k$类的样本子集。$n$是特征$A$取 值的个数，$K$是类的个数。\n",
    "\n",
    "（2）样本集合$D$对特征$A$的信息增益比（C4.5）\n",
    "\n",
    "\n",
    "$$g_{R}(D, A)=\\frac{g(D, A)}{H(D)}$$\n",
    "\n",
    "\n",
    "其中，$g(D,A)$是信息增益，$H(D)$是数据集$D$的熵。\n",
    "\n",
    "（3）样本集合$D$的基尼指数（CART）\n",
    "\n",
    "$$\\operatorname{Gini}(D)=1-\\sum_{k=1}^{K}\\left(\\frac{\\left|C_{k}\\right|}{|D|}\\right)^{2}$$\n",
    "\n",
    "特征$A$条件下集合$D$的基尼指数：\n",
    "\n",
    " $$\\operatorname{Gini}(D, A)=\\frac{\\left|D_{1}\\right|}{|D|} \\operatorname{Gini}\\left(D_{1}\\right)+\\frac{\\left|D_{2}\\right|}{|D|} \\operatorname{Gini}\\left(D_{2}\\right)$$\n",
    " \n",
    "4．决策树的生成。通常使用信息增益最大、信息增益比最大或基尼指数最小作为特征选择的准则。决策树的生成往往通过计算信息增益或其他指标，从根结点开始，递归地产生决策树。这相当于用信息增益或其他准则不断地选取局部最优的特征，或将训练集分割为能够基本正确分类的子集。\n",
    "\n",
    "5．决策树的剪枝。由于生成的决策树存在过拟合问题，需要对它进行剪枝，以简化学到的决策树。决策树的剪枝，往往从已生成的树上剪掉一些叶结点或叶结点以上的子树，并将其父结点或根结点作为新的叶结点，从而简化生成的决策树。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import math\n",
    "from math import log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_data():\n",
    "    datasets = [['青年', '否', '否', '一般', '否'],\n",
    "               ['青年', '否', '否', '好', '否'],\n",
    "               ['青年', '是', '否', '好', '是'],\n",
    "               ['青年', '是', '是', '一般', '是'],\n",
    "               ['青年', '否', '否', '一般', '否'],\n",
    "               ['中年', '否', '否', '一般', '否'],\n",
    "               ['中年', '否', '否', '好', '否'],\n",
    "               ['中年', '是', '是', '好', '是'],\n",
    "               ['中年', '否', '是', '非常好', '是'],\n",
    "               ['中年', '否', '是', '非常好', '是'],\n",
    "               ['老年', '否', '是', '非常好', '是'],\n",
    "               ['老年', '否', '是', '好', '是'],\n",
    "               ['老年', '是', '否', '好', '是'],\n",
    "               ['老年', '是', '否', '非常好', '是'],\n",
    "               ['老年', '否', '否', '一般', '否'],\n",
    "               ]\n",
    "    labels = [u'年龄', u'有工作', u'有自己的房子', u'信贷情况', u'类别']\n",
    "    # 返回数据集和每个维度的名称\n",
    "    return datasets, labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "datasets, labels = create_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = pd.DataFrame(datasets, columns=labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>年龄</th>\n",
       "      <th>有工作</th>\n",
       "      <th>有自己的房子</th>\n",
       "      <th>信贷情况</th>\n",
       "      <th>类别</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>青年</td>\n",
       "      <td>否</td>\n",
       "      <td>否</td>\n",
       "      <td>一般</td>\n",
       "      <td>否</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>青年</td>\n",
       "      <td>否</td>\n",
       "      <td>否</td>\n",
       "      <td>好</td>\n",
       "      <td>否</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>青年</td>\n",
       "      <td>是</td>\n",
       "      <td>否</td>\n",
       "      <td>好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>青年</td>\n",
       "      <td>是</td>\n",
       "      <td>是</td>\n",
       "      <td>一般</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>青年</td>\n",
       "      <td>否</td>\n",
       "      <td>否</td>\n",
       "      <td>一般</td>\n",
       "      <td>否</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>中年</td>\n",
       "      <td>否</td>\n",
       "      <td>否</td>\n",
       "      <td>一般</td>\n",
       "      <td>否</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>中年</td>\n",
       "      <td>否</td>\n",
       "      <td>否</td>\n",
       "      <td>好</td>\n",
       "      <td>否</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>中年</td>\n",
       "      <td>是</td>\n",
       "      <td>是</td>\n",
       "      <td>好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>中年</td>\n",
       "      <td>否</td>\n",
       "      <td>是</td>\n",
       "      <td>非常好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>中年</td>\n",
       "      <td>否</td>\n",
       "      <td>是</td>\n",
       "      <td>非常好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>老年</td>\n",
       "      <td>否</td>\n",
       "      <td>是</td>\n",
       "      <td>非常好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>老年</td>\n",
       "      <td>否</td>\n",
       "      <td>是</td>\n",
       "      <td>好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>老年</td>\n",
       "      <td>是</td>\n",
       "      <td>否</td>\n",
       "      <td>好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>老年</td>\n",
       "      <td>是</td>\n",
       "      <td>否</td>\n",
       "      <td>非常好</td>\n",
       "      <td>是</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>老年</td>\n",
       "      <td>否</td>\n",
       "      <td>否</td>\n",
       "      <td>一般</td>\n",
       "      <td>否</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    年龄 有工作 有自己的房子 信贷情况 类别\n",
       "0   青年   否      否   一般  否\n",
       "1   青年   否      否    好  否\n",
       "2   青年   是      否    好  是\n",
       "3   青年   是      是   一般  是\n",
       "4   青年   否      否   一般  否\n",
       "5   中年   否      否   一般  否\n",
       "6   中年   否      否    好  否\n",
       "7   中年   是      是    好  是\n",
       "8   中年   否      是  非常好  是\n",
       "9   中年   否      是  非常好  是\n",
       "10  老年   否      是  非常好  是\n",
       "11  老年   否      是    好  是\n",
       "12  老年   是      否    好  是\n",
       "13  老年   是      否  非常好  是\n",
       "14  老年   否      否   一般  否"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 熵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算给定数据集的熵（信息熵）\n",
    "def calc_ent(datasets):\n",
    "    # 计算数据集的长度\n",
    "    data_length = len(datasets)\n",
    "    # 统计数据集中每个类别的出现次数\n",
    "    label_count = {}\n",
    "    for i in range(data_length):\n",
    "        # 获取每个样本的标签\n",
    "        label = datasets[i][-1]\n",
    "        # 如果该类别不在label_count中，则添加到label_count中\n",
    "        if label not in label_count:\n",
    "            label_count[label] = 0\n",
    "        # 统计该类别的出现次数\n",
    "        label_count[label] += 1\n",
    "    # 计算熵\n",
    "    ent = -sum([(p / data_length) * log(p / data_length, 2)\n",
    "                for p in label_count.values()])\n",
    "    return ent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 条件熵"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算给定数据集在指定特征上的条件熵\n",
    "def cond_ent(datasets, axis=0):\n",
    "    # 计算数据集的长度\n",
    "    data_length = len(datasets)\n",
    "    # 使用字典feature_sets存储在指定特征上的不同取值对应的样本集合\n",
    "    feature_sets = {}\n",
    "    for i in range(data_length):\n",
    "        # 获取每个样本在指定特征上的取值\n",
    "        feature = datasets[i][axis]\n",
    "        # 如果该取值不在feature_sets中，则添加到feature_sets中\n",
    "        if feature not in feature_sets:\n",
    "            feature_sets[feature] = []\n",
    "        # 将该样本添加到对应取值的样本集合中\n",
    "        feature_sets[feature].append(datasets[i])\n",
    "    # 计算条件熵\n",
    "    cond_ent = sum([(len(p) / data_length) * calc_ent(p)\n",
    "                    for p in feature_sets.values()])\n",
    "    return cond_ent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9709505944546686"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "calc_ent(datasets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 信息增益"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算信息增益\n",
    "def info_gain(ent, cond_ent):\n",
    "    # 信息增益等于熵减去条件熵\n",
    "    return ent - cond_ent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用信息增益选择最佳特征作为根节点特征进行决策树的训练\n",
    "def info_gain_train(datasets):\n",
    "    # 计算特征的数量\n",
    "    count = len(datasets[0]) - 1\n",
    "    # 计算整个数据集的熵\n",
    "    ent = calc_ent(datasets)\n",
    "    # 存储每个特征的信息增益\n",
    "    best_feature = []\n",
    "    for c in range(count):\n",
    "        # 计算每个特征的条件熵\n",
    "        c_info_gain = info_gain(ent, cond_ent(datasets, axis=c))\n",
    "        # 将特征及其对应的信息增益存入best_feature列表中\n",
    "        best_feature.append((c, c_info_gain))\n",
    "        # 输出每个特征的信息增益\n",
    "        print('特征({}) 的信息增益为： {:.3f}'.format(labels[c], c_info_gain))\n",
    "    # 找到信息增益最大的特征\n",
    "    best_ = max(best_feature, key=lambda x: x[-1])\n",
    "    # 返回信息增益最大的特征作为根节点特征\n",
    "    return '特征({})的信息增益最大，选择为根节点特征'.format(labels[best_[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "特征(年龄) 的信息增益为： 0.083\n",
      "特征(有工作) 的信息增益为： 0.324\n",
      "特征(有自己的房子) 的信息增益为： 0.420\n",
      "特征(信贷情况) 的信息增益为： 0.363\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'特征(有自己的房子)的信息增益最大，选择为根节点特征'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "info_gain_train(np.array(datasets))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 利用ID3算法生成决策树"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义节点类 二叉树\n",
    "class Node:\n",
    "    def __init__(self, root=True, label=None, feature_name=None, feature=None):\n",
    "        self.root = root\n",
    "        self.label = label\n",
    "        self.feature_name = feature_name\n",
    "        self.feature = feature\n",
    "        self.tree = {}\n",
    "        self.result = {\n",
    "            'label:': self.label,\n",
    "            'feature': self.feature,\n",
    "            'tree': self.tree\n",
    "        }\n",
    "\n",
    "    def __repr__(self):\n",
    "        return '{}'.format(self.result)\n",
    "\n",
    "    def add_node(self, val, node):\n",
    "        self.tree[val] = node\n",
    "\n",
    "    def predict(self, features):\n",
    "        if self.root is True:\n",
    "            return self.label\n",
    "        return self.tree[features[self.feature]].predict(features)\n",
    "\n",
    "\n",
    "class DTree:\n",
    "    def __init__(self, epsilon=0.1):\n",
    "        self.epsilon = epsilon\n",
    "        self._tree = {}\n",
    "\n",
    "    # 熵\n",
    "    @staticmethod\n",
    "    def calc_ent(datasets):\n",
    "        data_length = len(datasets)\n",
    "        label_count = {}\n",
    "        for i in range(data_length):\n",
    "            label = datasets[i][-1]\n",
    "            if label not in label_count:\n",
    "                label_count[label] = 0\n",
    "            label_count[label] += 1\n",
    "        ent = -sum([(p / data_length) * log(p / data_length, 2)\n",
    "                    for p in label_count.values()])\n",
    "        return ent\n",
    "\n",
    "    # 经验条件熵\n",
    "    def cond_ent(self, datasets, axis=0):\n",
    "        data_length = len(datasets)\n",
    "        feature_sets = {}\n",
    "        for i in range(data_length):\n",
    "            feature = datasets[i][axis]\n",
    "            if feature not in feature_sets:\n",
    "                feature_sets[feature] = []\n",
    "            feature_sets[feature].append(datasets[i])\n",
    "        cond_ent = sum([(len(p) / data_length) * self.calc_ent(p)\n",
    "                        for p in feature_sets.values()])\n",
    "        return cond_ent\n",
    "\n",
    "    # 信息增益\n",
    "    @staticmethod\n",
    "    def info_gain(ent, cond_ent):\n",
    "        return ent - cond_ent\n",
    "\n",
    "    def info_gain_train(self, datasets):\n",
    "        count = len(datasets[0]) - 1\n",
    "        ent = self.calc_ent(datasets)\n",
    "        best_feature = []\n",
    "        for c in range(count):\n",
    "            c_info_gain = self.info_gain(ent, self.cond_ent(datasets, axis=c))\n",
    "            best_feature.append((c, c_info_gain))\n",
    "        # 比较大小\n",
    "        best_ = max(best_feature, key=lambda x: x[-1])\n",
    "        return best_\n",
    "\n",
    "    def train(self, train_data):\n",
    "        \"\"\"\n",
    "        input:数据集D(DataFrame格式)，特征集A，阈值eta\n",
    "        output:决策树T\n",
    "        \"\"\"\n",
    "        _, y_train, features = train_data.iloc[:, :\n",
    "                                               -1], train_data.iloc[:,\n",
    "                                                                    -1], train_data.columns[:\n",
    "                                                                                            -1]\n",
    "        # 1,若D中实例属于同一类Ck，则T为单节点树，并将类Ck作为结点的类标记，返回T\n",
    "        if len(y_train.value_counts()) == 1:\n",
    "            return Node(root=True, label=y_train.iloc[0])\n",
    "\n",
    "        # 2, 若A为空，则T为单节点树，将D中实例树最大的类Ck作为该节点的类标记，返回T\n",
    "        if len(features) == 0:\n",
    "            return Node(\n",
    "                root=True,\n",
    "                label=y_train.value_counts().sort_values(\n",
    "                    ascending=False).index[0])\n",
    "\n",
    "        # 3,计算最大信息增益 同5.1,Ag为信息增益最大的特征\n",
    "        max_feature, max_info_gain = self.info_gain_train(np.array(train_data))\n",
    "        max_feature_name = features[max_feature]\n",
    "\n",
    "        # 4,Ag的信息增益小于阈值eta,则置T为单节点树，并将D中是实例数最大的类Ck作为该节点的类标记，返回T\n",
    "        if max_info_gain < self.epsilon:\n",
    "            return Node(\n",
    "                root=True,\n",
    "                label=y_train.value_counts().sort_values(\n",
    "                    ascending=False).index[0])\n",
    "\n",
    "        # 5,构建Ag子集\n",
    "        node_tree = Node(\n",
    "            root=False, feature_name=max_feature_name, feature=max_feature)\n",
    "\n",
    "        feature_list = train_data[max_feature_name].value_counts().index\n",
    "        for f in feature_list:\n",
    "            sub_train_df = train_data.loc[train_data[max_feature_name] ==\n",
    "                                          f].drop([max_feature_name], axis=1)\n",
    "\n",
    "            # 6, 递归生成树\n",
    "            sub_tree = self.train(sub_train_df)\n",
    "            node_tree.add_node(f, sub_tree)\n",
    "\n",
    "        # pprint.pprint(node_tree.tree)\n",
    "        return node_tree\n",
    "\n",
    "    def fit(self, train_data):\n",
    "        self._tree = self.train(train_data)\n",
    "        return self._tree\n",
    "\n",
    "    def predict(self, X_test):\n",
    "        return self._tree.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "datasets, labels = create_data()\n",
    "data_df = pd.DataFrame(datasets, columns=labels)\n",
    "dt = DTree()\n",
    "tree = dt.fit(data_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'label:': None, 'feature': 2, 'tree': {'否': {'label:': None, 'feature': 1, 'tree': {'否': {'label:': '否', 'feature': None, 'tree': {}}, '是': {'label:': '是', 'feature': None, 'tree': {}}}}, '是': {'label:': '是', 'feature': None, 'tree': {}}}}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'否'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dt.predict(['老年', '否', '否', '一般'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scikit-learn实例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split\n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用Iris数据集，我们可以构建如下树："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# data\n",
    "def create_data():\n",
    "    iris = load_iris()\n",
    "    df = pd.DataFrame(iris.data, columns=iris.feature_names)\n",
    "    df['label'] = iris.target\n",
    "    df.columns = [\n",
    "        'sepal length', 'sepal width', 'petal length', 'petal width', 'label'\n",
    "    ]\n",
    "    data = np.array(df.iloc[:100, [0, 1, -1]])\n",
    "    # print(data)\n",
    "    return data[:, :2], data[:, -1],iris.feature_names[0:2]\n",
    "\n",
    "\n",
    "X, y,feature_name= create_data()\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 决策树分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9666666666666667"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.tree import export_graphviz\n",
    "import graphviz\n",
    "from sklearn import tree\n",
    "\n",
    "clf = DecisionTreeClassifier()\n",
    "clf.fit(X_train, y_train,)\n",
    "\n",
    "clf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一旦经过训练，就可以用 plot_tree函数绘制树："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Text(197.83636363636364, 195.696, 'X[0] <= 5.45\\ngini = 0.5\\nsamples = 70\\nvalue = [36, 34]'),\n",
       " Text(121.74545454545455, 152.208, 'X[1] <= 2.8\\ngini = 0.157\\nsamples = 35\\nvalue = [32, 3]'),\n",
       " Text(60.872727272727275, 108.72, 'X[0] <= 4.75\\ngini = 0.444\\nsamples = 3\\nvalue = [1, 2]'),\n",
       " Text(30.436363636363637, 65.232, 'gini = 0.0\\nsamples = 1\\nvalue = [1, 0]'),\n",
       " Text(91.30909090909091, 65.232, 'gini = 0.0\\nsamples = 2\\nvalue = [0, 2]'),\n",
       " Text(182.61818181818182, 108.72, 'X[0] <= 5.3\\ngini = 0.061\\nsamples = 32\\nvalue = [31, 1]'),\n",
       " Text(152.1818181818182, 65.232, 'gini = 0.0\\nsamples = 29\\nvalue = [29, 0]'),\n",
       " Text(213.05454545454546, 65.232, 'X[1] <= 3.2\\ngini = 0.444\\nsamples = 3\\nvalue = [2, 1]'),\n",
       " Text(182.61818181818182, 21.744, 'gini = 0.0\\nsamples = 1\\nvalue = [0, 1]'),\n",
       " Text(243.4909090909091, 21.744, 'gini = 0.0\\nsamples = 2\\nvalue = [2, 0]'),\n",
       " Text(273.92727272727274, 152.208, 'X[1] <= 3.5\\ngini = 0.202\\nsamples = 35\\nvalue = [4, 31]'),\n",
       " Text(243.4909090909091, 108.72, 'gini = 0.0\\nsamples = 31\\nvalue = [0, 31]'),\n",
       " Text(304.3636363636364, 108.72, 'gini = 0.0\\nsamples = 4\\nvalue = [4, 0]')]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA+30lEQVR4nO3de1xVVd748c9GwOMNNS/jCGoxlj6jSI6Xx/ExIzTTp0TDG2MgBoIQKkczTDRkvCFQoiRFXBL9NYSZlaU9jpZp5TOYmakjoZgHH2HURMlAuR7W7w/kjCB3DufGer9e+6WcfVtf9j6Lvb977bUUIQSSJEmSYVgZuwCSJEltiax0JUmSDEhWupIkSQYkK11JkiQDkpWuJEmSAclKV5IkyYBkpStJkmRAstKVJEkyIFnpSpIkGZCsdCVJkgzI2tgFkKTm6tChw7Xi4uLfGbschqRSqa4XFRX1MXY5pOZTZN8LkrlSFEW0tfNXURSEEIqxyyE1n0wvSJIkGZCsdCVJkgxIVrqSRTpy5AhRUVEAeHl5kZeXh5ubG2lpaQBER0ejVqt55ZVXADh48CDjxo2rd5vFxcUN7jc7O5sRI0YQEBDAJ5988sD8w4cPM3ToUF0ZXVxcCAgI4JtvvmlKeJIZk5WuZJFcXFy4fv06K1aswM3NjZ49e2JnZ4eHhwelpaVkZmayZcsWOnbsyKVLl5g0aRIPP/zwA9spKSlh165d+Pr6snv37kbtu1OnThQVFfHII49U+7ywsJAvvviCxx9/HKjMz3bs2JHS0lL69+/f0pAlMyFbL0gWa/bs2cyaNYuIiIhqn9+8eZNevXoB4ODgQG5uLo6Ojg+sHx8fz/79+wkKCiIxMRErq8prlBUrVnD79m3dcgsWLGDkyJEADBgwgK+//po7d+7g4+PDrl27dMtFRUWhVqtZtmwZAE888QRPPvkkV69eJSwsjMTERP3+AiSTJK90JYuk1WqJiYkhNjaWLVu2VJvXo0cPbty4AUBOTg729va1bsPNzY2xY8fy4YcfkpKSQkFBQYP7VZTKhgUdO3akZsuK06dPs3HjRk6ePMnu3bt1lXj37t0blbqQLIO80pUs0htvvMHChQt56qmnWLJkCVlZWbp5tra2DB48GLVajY2NTa1XuQB9+/Zl5cqVAHz99dfs378fDw8PIiMj69zvsWPH2L59O8XFxfzlL38BYP78+aSkpLB3714APD09mTVrFnv37uXzzz/n9u3bLFmyRF+hSyZOttOVzFZT2+l6enry3nvvNXu+KZDtdM2fTC9IbYa9vb2u9UJNBw8epEOHDgYukdQWyStdyWzp6420VatWsWHDhkZ/Xpfo6Ghyc3OxsbEhOjpa93lKSgp79uzB3t6eJUuW8Mc//rHZZZVXuuZPXulKbcpPP/3E3LlzCQsLw93dHYDLly+TnZ2Nq6srUVFRLF26VPd5Y9XWDK2KlZUVKpUKgN///vd6jEYyR/JBmtSmJCUlERMTw0MPPaSrdKs4OzsTEhKCl5dXresmJydz4sQJ3c/jx49n7ty5QP3N0Dw9PZk3bx4//PADMTExrF27tjVCk8yEvNKV2hQhBIqi6Jp23a9Tp066ZZqqvmZoVU3Devfu3ahmZ5Jlk1e6Upvi5+fHsmXLcHR0pHPnzk1a19fXF19f31rn1dYMLSIiAnd3d7755hu+//57bt26xfr16/URhmTG5IM0yWw150FaXl4esbGx5OXlMXXqVKZMmdJKpWsd8kGa+ZOVrmS2ZH+6kjmS6QVJaoA+X5o4cuQIaWlpFBQUUFFRwfvvv19nUzPJMslKV7JIqampHD16FDs7O6Kjo9m6dSsajYaePXuyevVqZsyYwZAhQ8jIyMDV1ZXjx48zc+ZMnJycmDdvHu7u7mg0GrZu3arbZlxcHFlZWRQWFrJ582aWL1+OnZ0dI0eOxMPDo1HlcnFxwcXFhcTEROzt7XVNzZKTk1mzZg2XLl2q87VkyTLI1guSRcrJycHJyUnX5raiooLOnTvz6aefIoSgoqKC8PBwJk2aRJ8+fUhMTNT1f+vk5IRarcbBwYGzZ88Cld0ypqWl0a1bN9q3b09mZib5+fm4urpWa3pWXl5OQEBAtUmj0TxQvr///e8888wztTY1kyybrHQlixQSEsLYsWMJCgoiJyeHrKws1q9fT48ePdBqtXTq1AkrKytsbW2xs7PD1taWkpISAMrKyqr9C5XNyAYMGEB4eDhxcXGMHj2aHTt2UF5ejp+fX5PK9tNPPzFo0CDatWvX6B7PJMsh0wuSRUpISCArKwuVSkX37t3Jz88nOjqaq1evNrhuZmYmK1eu5NatWzg5OQHQpUsXhgwZQnBwMKWlpYSGhhIREYFKpWLQoEG6da2trYmPj693+9u3b9dV1I3t8UyyHLL1gmS2WqP1QnZ2NklJSSbbnla2XjB/stKVzJZsMiaZI5nTlSRJMiBZ6UoW78iRIyQlJel1m8OGDSM9PZ3s7GwCAgKYPXs2CQkJQOUoE4sWLSI4OJjCwsIH1j158iSBgYFMmzaNzz77TPf5G2+8gaenJ9C40Ykl8yQrXcnsBQYGUlBQwJ07dwgMDOTs2bOsXLmS+fPnc/78ed1yKSkpfPHFFwC6ym3dunWo1WoWLVqEVqtt9D6HDRvGmDFjePjhh4mPjyctLY1Tp04Ble15O3bsSPfu3XWd6NxvxIgRvP3226SkpHDs2DEAfvzxR7p3765bpq7RiSXzJytdyexNnz6dvXv3snfvXtzc3Gjfvj0lJSXY2dmxZ8+eOtfLyMggPT2dbt26UVRUVK2NbHJycrW2tqmpqXVu58CBAzz55JM89dRTAJw5c4ZNmzbRr18/Dh48WOs6qampPPPMMzz77LOUlZWxfft2XnzxxWb+BiRzIitdyexNnDiRw4cPc+jQIZ5++mliY2N59dVXeeGFF7h7965uOVtbW8rLyxFCUFRUREVFBc7OzoSHh5OcnEz//v2btf/JkyfzzTff8MEHHwDwH//xH1hZWdG9e/c6u3KcO3cux44dIy4ujnPnznHz5k2WLl3KyZMndVfMkmWS7XQls9euXTu6du2KVqvF2toaFxcXNmzYgI2NjW7EBqjsdDw0NJRz586h1WoZOnQoxcXFvPzyyxQUFBAbG6tbvr5uHO/3/fffs3PnTkpKSpgwYQIAM2fOJCgoiDt37vDWW29x6NAhevfujbOzM1CZr923bx937txh9uzZPP7447q+HTw9PRk+fLi+f0WSCZFNxiSzZcwmY03pBCcqKoqXXnqpyf331rYP2WTM/Mn0giQ1Q1lZGenp6Y1aNiQkpMkVrhyd2HLJK13JLCmK0hu43tbOX3mla/5kTlcyK4qitAcWA6+qVKq7iqJ0NHaZDEmlUhUqitJRCHG34aUlUyTTC5JZUCpNA84BLsB/FRUVdRJCKG1lAh4pLi7+HMhUFOUvSm2ja0omT6YXJJOnKMowIAboAywTQvzdyEUyKkVRngC2ACWAWgjxnXFLJDWFvNKVTJaiKL0VRXkH+AL4CHBu6xUugBDiG2AUkAB8rCjKTkVRZEe8ZkJWupLJURTFVlGU5UAGcBcYJISIE0KUG7loJkMIUSGESAEGA1eAM4qihLW1HLc5kpWuZHSKovRVFGVYbXlbIcRSIUS+cUtouoQQBUKIVcAIYAjwU1W+V1GUQYqiPGLkIko1yJyuZFSKorQDvgYOAeOA3yPzts1WI9/7CeAFjBBClBqxWNJ9ZKUrGZWiKGFAANAO2AjINEILKYpiBfgAfwW0wEdCCLVRCyXpyEpXMpp7V7lVoz9WAP8rhBhvxCJZDEVR9gL/TeUfMwXoJYTIM26pJJCVrmRkiqI8BBQDRW1u7J1Wdq8drwroKIS4aezySJVkpWuCOnTocK24uPh3xi5Ha1GpVNeLior6GLscbZU8v4xLVromyNIHXJT9BxiXPL+MSzYZkyRJMiBZ6UqSJBmQrHTNzJEjR4iKigLAy8uLvLw83NzcSEtLA2DRokWMGjVKt3xjRpUtLi5ucL9ffvklfn5+TJs2TTe4Y5XExERefPFF3N3duXz5clNDkkyMsc6x2kZWrjJ//nwWLlxIQEAA5eXm3aJQVrpmxsXFhevXr7NixQrc3Nzo2bMndnZ2eHh4ALBt2zYGDRqkW76uUWVLSkrYtWsXvr6+7N69u8H9TpgwgcTERHbs2MFHH31UbV56ejoJCQnMmDGD06dPtyxAyeiMdY7VNrJylaoO3Xv37o21tXn3SGvepW+jZs+ezaxZs4iIiGjW+vHx8ezfv5+goCASExOxsqr827tixQpu376tW27BggWMHDmy2roRERH4+flV+8zNzY2JEydSUVHBvn37mlUmybQY6xw7cOAAGzZsYPHixdW2FxcXh5WVFZs3b+bw4cO4uro2q1ymQF7pmhmtVktMTAyxsbFs2bKlWdtwc3Nj7NixfPjhh6SkpNQ5Ym1N4eHhTJgw4YGBE9977z2OHj3K66+/zrvvvtusMkmmw5jnWM2RlatUVdq9e/du9LZMlhBCTiY2VR6W2kVGRorDhw8LIYRYvHixuHDhgnjhhRd089esWSMeffRR8dJLL+k+u39+TUePHhXvv/9+nfOrvPfee2LYsGFi4cKF4p133hFCCOHt7S2EECI6Olr4+fmJGTNmiFOnTjW4rXvxGf333Fan+s4vIYx3jp04cUIsXrxY+Pv7i7feeksI8e9zbPny5SIwMFB4enqK4uLierdj6ueX0Qsgp1oOSgNfiprqO+EbM9/QTP1LYelTU88vIczrHDP180umFyyAvb297slyTXJUWUkf5DmmR8au9eX04EQTr0RCQ0Ob9HldoqKiRHBwsFi+fHm1zw8ePCimTJkiEhMTdZ89+uijYuHChSIpKUkIIcTGjRvFwoULxYgRI8SBAwfq3Q8mfiVi6ZOpnV/btm0TCxYsEM8++6zIzc0VQggREhIigoODRUREhBCiMs0xf/58MW/ePKHVauvdj6mfX/JK18z89NNPzJ07l7CwMNzd3QG4fPky2dnZuLq6EhUVxdKlS3WfN1ZpaSmZmZls2bKFjh07cunSJd28p59+mpCQkGrLd+zYkaKiIh55pLKP7JUrVxIfH88jjzzCxIkTWxqmZCTGOL+qWjgsWLCAb7/9lsuXL2Ntbc2WLVv4+eefKS4uJjY2lu3bt9O9e3fy8sy7szTZZMzMJCUlERMTw0MPPaT7UlRxdnYmJCQELy+vWtdNTk7mxIkTup/Hjx/P3LlzAbh58ya9evUCwMHBgdzcXBwdHessxw8//EBFRQUzZ87UNd85f/48AwcOpF27di2KUTIeY51fd+/eZc+ePbz99tucPXuWfv36AZWtFW7cuEG/fv3IysqirKyM3r176zVmQ5NXumZGCIGiKNQ2+nanTp10yzRVjx49uHHjBgA5OTnY29c/zqGVlRXW1tbY2NjoPtu+fTvz589v8r4l02GM8+vu3bssWrSIDRs20LlzZ/r27UtOTg4AN27coFevXly8eJGoqChiYmKaE5ZJkVe6ZsbPz49ly5bh6OhI586dm7Sur68vvr6+tc6ztbVl8ODBqNVqbGxscHR0JCIiAnd3dwoKCti8eTO//fYb9vb2DBw4kI0bN6Ioiu4qV6vV8vPPP1d7U0kyP8Y4v15//XWuXr3Kxo0b8fDwwMXFhbKyMpYuXYqjoyMqlYrnnnuOMWPGsGTJEsLCwujbt68+wjUK2bWjCaqv6728vDxiY2PJy8tj6tSpTJkyxcClazlT73rP0snzy7hkpWuCZH+nUmuS55dxyZxuG+Tp6am3bZ08eZLAwECmTZvGZ599BsCf//xnAgICdD1VSW2LPL/qJ3O6ZiI1NZWjR49iZ2dHdHQ0W7duRaPR0LNnT1avXs2MGTMYMmQIGRkZuLq6cvz4cWbOnImTkxPz5s3D3d0djUbD1q1bdduMi4sjKyuLwsJCNm/ezPLly7Gzs2PkyJG6HqUaMmLECEaMGEF+fj6RkZFMnTqVjh07UlpaSv/+/Vvr1yHpmTy/DEde6ZqJnJwcnJycdG0kKyoq6Ny5M59++ilCCCoqKggPD2fSpEn06dOHxMREPvnkEwCcnJxQq9U4ODhw9uxZAAoLC0lLS6Nbt260b9+ezMxM8vPzcXV1rdZUqLy8nICAgGqTRqOpVrbU1FSeeeYZnn32WQAOHTrEu+++y8cff0xRUZEBfjtSS8nzy3BkpWsmQkJCGDt2LEFBQeTk5JCVlcX69evp0aMHWq2WTp06YWVlha2tLXZ2dtja2lJSUgJAWVlZtX+hstnPgAEDCA8PJy4ujtGjR7Njxw7Ky8sf6LqxIXPnzuXYsWPExcUB/+4Rys7OjtLSUn2EL7UyeX4ZjkwvmImEhASysrJQqVR0796d/Px8oqOjuXr1aoPrZmZmsnLlSm7duoWTkxMAXbp0YciQIQQHB1NaWkpoaCgRERGoVKpqzb6sra2Jj4+vc9sHDx5k37593Llzh9mzZ/Pbb78RFBREhw4d6Nu3L127dm158FKrk+eX4cjWCyZIn0+Xs7OzSUpKYv369XrZnj6Y+tNlSyfPL+OSla4Jkk16pNYkzy/jkjldSZIkA5KVrhk7cuQISUlJet3msGHDSE9Pf2Bk1rKyMubPn8+8efN4+eWXa123tjaVoaGhrF69Wq9llAynNc8xqHzgNm3atDr3sW3bNt0o1L/++iunTp1izpw51c4pczvHZKVrwgIDAykoKODOnTsEBgZy9uxZVq5cyfz58zl//rxuuZSUFN2w6FUN09etW4darWbRokVotdpG73PYsGGMGTPmgZFZbWxsSElJYefOnXU+XBkxYgRvv/02KSkpHDt2DAB/f//mhi8ZgDHPMYDExMR6uwJdtGgRiYmJPPXUU1y9epXhw4cTGRlZbRlzO8dkpWvCpk+fzt69e9m7dy9ubm60b9+ekpIS7Ozs2LNnT53rZWRkkJ6eTrdu3SgqKiI3N1c3Lzk5uVqbyNTU1Dq3c+DAAZ588kmeeuop3Wf/+7//y2OPPVbnOjXbVEqmzZjnmEaj4ddff9W1eKiLv78///M//4ODg0PzgjQxssmYCZs4cSJ+fn4IIUhMTEStVhMWFoZGo9HdvkNlD07l5eUIISgqKqKiogJnZ2fCw8NbtP/JkyczefJkZs6cyezZszl+/Dgffvghb7zxRp3rzJ07l1mzZuHl5cUTTzzRov1Lrc+Y59hXX31FdnY23333Hbdv32bOnDl06dLlgeUSEhLYvXs3n3/+OXPmzGn2/kyFrHRNWLt27ejatStarRZra2tcXFzYsGEDNjY2qFQq3XLjx48nNDSUc+fOodVqGTp0KMXFxbz88ssUFBQQGxurW76+7vfu9/3337Nz505KSkqYMGECd+/eZdq0abi5uREYGMjWrVv5+uuv6d27N87OzsCDbSol02fMc8zHxwcfHx+OHDnCxYsX6dKlC9HR0bzyyiu6ZdatW8eNGzfIz89n8+bNaDQaVq9eTWZmJoMHD9ZrPw8GY+zxguT04EQzRmvVl6aM6hoZGSkKCgrqXUaj0YhVq1ZV+wwTH8PK0idjnl9C1H+Ovfbaa03eXs1zzNTPL5nTlaopKyvTPVluSEhISIMdXe/cuVM3jpokQf3n2Nq1a5u8PXM7x+TLESZINl6XWpM8v4xL5nRNkEqluq4oyu+MXY7WolKprhu7DG2ZPL+MS6YXTIyiKEpxcfF6IA/wBayEEIo5T1T+cX8VuAF4FBUV9THir7jNKyoq6tOMY2gFHADWGuB8mQDkAj2bs76pn18yvWBCFEXpDbwL/A6YK4TIMnKR9EpRlBFAKvAPYLEQosDIRZIaSVGUIMAb+C8hRFlDy+thf28AA4BZlpYLkVe6RqRU+q97/58M/AicAcZaWoULIIQ4CfwJKANOKYryn/f/DiTTpCjKYCAc8DREhXvPKuAxYJ6B9mcw8krXiBRF+W8gAvgKcAfmCSGOGLVQBqIoygzgLeBNIAiYIoT40aiFkh6gKIotlXcmCUKIdwy872HAl8BoIYSmoeXNhax0jURRFCsgA2gPnAUChRC59a9lWe5dQSUCvwcuCyEmGLlIUg2KomwAhgFuxrjNVxRlOTANcBFCNL6DBxMm0wvGswwYRGX+9s+Ai1FLYxzPAk6APeCqKMpUI5dHus+9tI8PsMCIedXNQDnwSkMLmgt5pWskiqL0pzK/+a0QIs/Y5TEWRVEUKv/wjAcOCSHyjVykNk9RlADgCpWpH7UQ4lMjl6c/8D0wC/AQQgQaszwtJStdSZKqURTlf6hs5vcvYIUQ4pqRy9MJeBFYBPQHOgshKoxZppYwm5cjOnTocK24uNhiGnSrVKrrpt6e0JAs7fjWZGbHezigorKVyTEgwbjFYRCVrSfygXZUpqOuGLNALWE2V7qW9uqiqb+qaGiWdnxrMpfjfS/dowW+AQKEED8ZuUgAKIrSA/grlS1dPIQQu4xcpGaTla6RmMuX0FAs7fjWZE7HW1GUAUKIy8YuR20URekH5JpzekFWukZiTl9CQ7C041uTPN5SFbNsMnbkyBGioqIA8PLyIi8vDzc3N9LS0gCIjo5GrVbrOkM+ePAg48aNq3ebxcXFjdq3ELUPpBcREUFAQAAjR47k73//O0eOHMHFxYWAgAC++eabpoYo3cdYxzs7O5sRI0YQEBDAJ598Um1ezQETJamxzLLSdXFx4fr166xYsQI3Nzd69uyJnZ0dHh4elJaWkpmZyZYtW+jYsSOXLl1i0qRJPPzwww9sp6SkhF27duHr68vu3bsbte+6BtJbuXIl8fHxPPLII0ycOBFFUejYsSOlpaX079+/pSG3acY83p06daKoqOiB/lprDpjYVB06dLimKIqw5KlDhw61tnqwxNjrirU2ZtN6oabZs2cza9YsIiIiqn1+8+ZNevXqBYCDgwO5ubk4Ojo+sH58fDz79+8nKCiIxMRErKwq//6sWLGC27dv65ZbsGABI0eOBP49kN7o0aO5ePHiA9s8f/48AwcOpF27djzxxBM8+eSTXL16lbCwMBITE/UWe1tkjOM9YMAAvv76a+7cuYOPjw+7dlV/duPv78/ly5cbNTRNTcXFxb+z5HQKgFJH95GWGHtdsdbGLK90tVotMTExxMbGsmXLlmrzevTowY0bNwDIycnB3t6+1m24ubkxduxYPvzwQ1JSUigoaLjDq6qB9LZt28auXbseWGf79u3Mnz8fQPel7t69e6NTF1LtjHW8Kx/kQ8eOHamtkkhISGDBggV8/vnnTYxIatOECYwZ1JiJ+8Z1ioyMFIcPHxZCCLF48WJx4cKFauMuRUVFieDgYLF8+XLdZ/WNy3T06FHx/vvv1zm/pq+++kokJiYKIYTw9vYWQghRXl4uZs6cqVvmk08+Ef7+/mLOnDni2LFjD2wDEx/HydAT9YzbZazj/e233wpfX1/xwgsviI8++kgI8e/jvXbtWrF48WLh6ekpfvnllwa3VfN41xdvY4WGhjbp87rU9vsTQogrV64IT09PMW/ePPHDDz80uXx1neMtid1UY27K99noX7ZGF7SBA9XQgIpNGXDREGSl27Ivorkf76bGm5GRIf7yl7+I1157TTz//PNCiMoYNRqNeOqpp0RkZKRQq9W6zxurpKRE+Pj4CCGECAsLEz///LNu3l//+ldx4cIFcffuXeHn59ek8gpR9zne2NjNKeamfJ/NNqdbk729PWlpaXh4eDww7+DBg3To0MEIpZJaS1s73klJScTExPDQQw/h7u5ebZ6zszMhISF4eXnVum5ycjInTpzQ/Tx+/Hjmzp0L1J8Tz83NpV+/fqhUKqOkyCw1ZrPM6dYmMjISDw8PVq1a9cC8SZMm0bt37yZtr2YzpPvduHGDgQMHVnuYtnjxYlavXq37+fDhwwwdOrRJ+5Qaz9rautYKd9WqVUyaNKnRDy7rOs45OTl4eXnh7e3NqVOngMqWK4sXL2bTpk1AZbOxP/3pT7U+VNU3IQSKoujyzPfr1KmTbpmmqi8nbm9vz5UrVygqKkKlUjWz5M1nqTGb9ZXuTz/9xLp16xg4cCD//Oc/+eijj7h8+TLZ2dn4+PgwefJkrl69SkxMDJcvN/4Fm6pmSMnJyaxZs4ZLly5VeyL++uuvV/vLu3//foYOHcqVK5WvgxcWFvLFF1/w+OOP6y3WtswYx/ndd98lLCwMBwcHgoODWbduHR9//DHDhg2jb9++QGWzsbw8w3QQ5+fnx7Jly3B0dGxw2PuafH1962xhYWtry+DBg1Gr1djY2ODo6EhERATu7u74+PgQGhqKlZUVwcHB+gijSSw1ZrOudI1x+7F7924mTZqke+Hh119/5dtvv2XhwoW6FyaioqJQq9UsW7ZMvwG3UaZwm6nRaOjVqxebNm3C19eX559/ni5durRGuLXq1asXjo6O5OXlMW9e5Qg27733HgDr16+v9nPVv41V8yp/5cqVuv/v3Lmz2WVuKUuN2awrXUPdfkyY8O8BDdLT09FqtaSnp1NUVISLiwu//PILa9eu5dy5c/j5+XH69Gk2btzIyZMn2b17N7NmzWpmhBIY5zhX3WY6ODigUqno27cv3bt31+2zpKTEoJVuz549Wbt2rcH2ZwosNWazrnSNcfvxxhtvABAeHo6npycDBw5kypQpZGdnk5SUxIABA9i7dy8Anp6essLVA1O4zezfvz8dO3Zk2bJldO7cmZ49e5Kamsq+ffu4cOEC69at4w9/+IM+wtULT0/PJl/91eXkyZMkJSXxr3/9iwULFjB16lQWLVrE8ePHq91FmAp9xg6Vr4w/+eSTbNiwoda3UZvKrDu8ycvLIzY2lry8PKZOncqUKVOMVLqmkx2gVFdfhzfmfJyr1DzetcWbmprK0aNHsbOzIzo6mq1bt6LRaOjZsyerV69mxowZDBkyhIyMDFxdXTl+/DgzZ87EycmJefPm4e7ujkajYevWrbqKJy4ujqysLAoLC9m8eTPLly/Hzs6OkSNH1vogsj75+flERkbqHiQ2VLnVdY6bW+wbNmygXbt2jBw5ss5KtynfZ7O+0rXU2w+purZynHNycnByctLlrSsqKujcuTOffvopq1atoqKigvDwcJKSkujZsyeJiYkEBgbi5OSEk5MTarWa6Ohozp49C1Q+0E1LS2PChAmUlZWRmZlJfn4+06dPr1Z5lJeXs2jRomplWbFiRbX+JlJTU9myZYvuTq+txJ6eno6Dg0Oz0ld1sZgmYy3l6empt22dPHmSwMBApk2bxmeffaa37UrN05rH9l//+hc+Pj7MnTuX6OjoFm07JCSEsWPHEhQURE5ODllZWaxfv54ePXqg1Wrp1KkTVlZW2NraYmdnh62tLSUlJQCUlZVV+xcq89wDBgwgPDycuLg4Ro8ezY4dOygvL8fPz69JZZs7dy7Hjh0jLi6uRTHWxVRjP3z4MKdOnSI1NVVv/aeY9ZWuPm9JqujjlmTEiBGMGDFCdzs2daoc5LapzOnYvvvuuwC6VhHNlZCQQFZWFiqViu7du5Ofn090dHSjejHLzMxk5cqV3Lp1CycnJwC6dOnCkCFDCA4OprS0lNDQUCIiIlCpVAwaNEi3rrW1NfHx8XVu++DBg+zbt487d+4we/ZsoPKZxnfffUdQUJBeKmJTjT00NBSAlJQUHBwcWhjlPY19dc3YE7W8OhgZGSnefPNNkZubK4QQYvPmzWLVqlVi1KhRoqKiQkyfPl1otVrxzjvviD179uhe/9NoNOKll14SQlS+g33mzBnxwgsviIKCAjFu3DixZs0a8dJLL4njx4+LmTNniv3794uSkhLdfsvKysTChQurTZcuXapWtr/97W9i1KhR4uuvv36g3PcSWsLYv1NTmmoeX3M7trt37xbvvPNObYdaCPHg8a7tfG4ujUYjVq1apbft6Utd57glxt6U77NZpxdM9ZYEWv92zNKZ07H99NNPuXjxIv7+/nqKvmkefvhhXbvVtsYcYzfr9IKp3pLUdjsmNY25HFuNRkNAQABubm4sX76c119/veXB13DkyBEuXrzIggUL9LbNYcOGkZCQwOjRowkMDMTGxoYRI0bw4osvPrDstm3bOH36NL/88gs7duxAo9GwadMmHn30UV2FV9W8Tt8VYGvGPmbMGIQQTJ8+nalTp9a6j0OHDpGWlkZ5eTlvvvkm165dw9/fn6SkJAYOHNi8AjT2ktjYExZ2S4JML7TK8TWFY1ubmsf7/ngDAgLEb7/9JgoLC0VAQIA4c+aMePXVV4W3t7fIzMzUdSW6fft2cejQISHEv3vVWrt2rQgODhZBQUGivLy80eWpWv/o0aMiISGh2md1iYmJERkZGUKIB3/Ptf3e6zrHTSV2IYR45513RGxsrK6r1po8PT1FRUWF+PLLL0VycrIQQog1a9aIrKysRsVa22TWV7rNZY63JFLjmOOxnT59Onv37sXKygo3Nzfat29PSUkJdnZ27Nmzh7Fjx9a6XkZGBunp6YwaNYorV66Qm5urGxqqvtef71f1ujNAu3bt6ixjS0bJqI8xY29oJBhA9yZkv379OHbsmB4iNvP0giRZgokTJ+Ln54cQgsTERNRqNWFhYWg0mmpNDm1tbSkvL0cIQVFRERUVFTg7OxMeHt7sfdvb23P+/Hmgsm1sXRISEti9ezeff/45c+bMafb+ajJm7FUjwXz33Xfcvn2bOXPm1Plqd32jkjSVRVa6rZkHsre3Jzw8nJKSkjrfxlmyZAkFBQVUVFSwfft23UOWFuWBJKB1j22fPn3YtGkTt27dYuLEifj7+7NmzRref/99MjIysLZ+8Oty6tSpFuc327VrR9euXdFqtVhbW+Pi4sKGDRuwsbGp1r3g+PHjCQ0N5dy5c2i1WoYOHUpxcTEvv/wyBQUFxMbG6pav7/Xn+40bN47U1FROnz6Nq6srUNnd5f0dwqxbt44bN26Qn5/P5s2b0Wg0rF69mszMTAYPHtyidtDGjN3HxwcfHx/dOdWlS5cHYvfy8mLBggWUl5cTGxvb7DiraWwewtgTJpQHqu3n2gQHB4vr168LIR7MAyFzurUeX1M4tlqtVgQEBOh+9vb2FmVlZXWu35z8Jnp8RtEc9Z2/r732WpO319ycrjG0NPY2mdM1Zh6osbKysigrK2ty5+ltnbGP7YEDB9iwYQOLFy/Wf3AmpKysjPT0dMaMGfPAvOa8cr1z584Hhqk3VS2J/cKFC2RnZ7doZBKzrHSNmQdqjIsXLxIVFSXb6DaDsY/t5MmTmTx5MjNnzrTo5n41h5NvqbCwML1urzW1JPbHHnuMlJSUFu3fLCtdY+aBfvvtN0JCQvjuu+/YunUrwcHBD+SBnnvuOcaMGcOSJUsICwvTjTQgNcyYx/b7779n586dlJSU6PrWffPNN/nHP/5BUFAQkZGRnDhxgt69e+Ps7AzQ7PymSqW6rijK75r6+zEnKpXqel2fW1rsdcVaq8bmIYw9YWF5IGRO12SOb1NGko2MjBQFBQX1LtOU/Kac2t5k1q8BG1JVHqg2hsgDSa2nvmNbU0hISIMdqZtTflMyPLPuxNycyU7Mq7O041uTPN5SFbPJ6VpaHqhJOaA2wNKOb03yeEtVzOZKt6kURQkGPIAnhBDlrbifVOCmEMKy2xiZIKVypMqPgfNCiBWtuJ/OwI9AiBDio9baj9Q2WGSlqyjKEOAI8GchRO0vVetvX92B04C/EOJAa+5Lqk5RFF9gMfCfQoiSVt7XGOATYLgQouGuziSpDhZX6SqKYgscB+KEEEkG2qcr8P8AZyFEniH22dYpijIQSAeeFEKcM9A+/wqMBjKBD4UQ+ukBRWpTLLH1wlrg/4BkQ+1QCHEYeB94594tr9SKFEWxpvKP3DpDVbj3fA30Ah4H/tOA+5UsiNk8SGsMRVHGA/OAx43wKHwVcALwBlIMvO+2ZiVQCLxp4P1OBQYAXQH5YExqFotJLyiK0pXK3GqQEGK/kcrgBBwGRgshNMYog6VTFGUUsA/4kxAi1wj7dwbeAzoIIWSXcVKTmX2lqyjKI8BNYBtwRwgRaOTyvAw8D0wA/iCEyDBmeSzFvcruIvAD8JoQ4gMjl8dKCFF3B7SSVAdLqHQ/BX4CplN59XPHyOWxAr6g8qr7GSHEH41ZHktwL09+G9gFqIQQXkYukiQ1myU8SPsj4Eflk+ypRi4LwDggh8rcsqOiKHWPgSI1Vh9AC0wB2imKIv+QSWbLrB+k3XuK7QiUAKXAV8YtEVB5+5sL2ALtgceovBKXmm8U0A1QgDNUphn0qkOHDteKi4st5o04lUp1vaioqI+xyyE9yKzTC4qi2ABHqXwx4Z/GLs/9FEVxoLJZ00IhxAVjl8ec3WsHvRSYL4S42Ur7sKi+H2RfD6bLrCtdSdIXWelKhmIJOV1JkiSz0eRKt0OHDtcURRGWMnXo0OFaW4q3vpgtMdamHu+ajhw5QlRUFFA5MmxeXh5ubm6kpaUBsGjRIkaNGqVb/uDBg4wbN67ebRYXFze43+zsbAICApg9ezYJCQnV5i1ZsoQXX3wRb2/veodNl0xUU3s9x8gjOOgbDfTob2nxClF3zJYYa03NiX3ZsmUiJCREfPDBB0KIhkeGrm0kiuLiYpGWliZ8fHzEzp07G13emiMT3+/+0aZraui8lpPxJrNuvSBJhjB79mxmzZpFREREs9aPj49n//79BAUFkZiYiJVV5Q3mihUruH37tm65BQsWMHLkSN3P9Y1MLEebNl8Gy+muWrWqSZ/XJTo6GrVaXW0gSICcnBy8vLzw9vbm1KlTzS6nPsmYG/68LqYSs1arJSYmhtjYWLZs2dKsbbi5uTF27Fg+/PBDUlJSKCgoaNR6kydP5ptvvuGDD6q/fFc12nRMTEyzyiMZWVMvjWnELWhGRob4y1/+Il577TXx/PPPCyEqb7k0Go146qmnRGRkpFCr1brPG6ukpET4+PgIIYQICwsTP//8s27eX//6V3HhwgVx9+5d4efn1+htoqf0giXE3NhYq5hTzFWaGntkZKQ4fPiwEEKIxYsXiwsXLlSLZc2aNeLRRx8VL730ku6z+mI9evSoeP/99xss54kTJ8TixYuFv7+/eOutt4QQQnh7ewshhBg0aJDw9vYWCxcuFLm5uU2KU07Gn1olvZCUlERMTAwPPfQQ7u7u1eY5OzsTEhKCl1ftb3ImJydz4sQJ3c/jx49n7ty5ANy8eZNevXoB4ODgQG5uLo6OjgDk5ubSr18/VCpVox5U6JuM2TJjDgkJ0f0/Njb2gfnh4eGEh4c3envjx49v1HIjR46slmoASElJASAzM7PR+5NMT6ukF4QQKIqCUkvXsp06ddIt01Q9evTgxo0bQOVtpr29vW6evb09V65coaioCJVK1cySN5+MuTpLjbmqDFWtF2o6ePCgHPVZqlerXOn6+fmxbNkyHB0dGxyuuiZfX198fX1rnWdra8vgwYNRq9XY2Njg6OhIREQE7u7u+Pj4EBoaipWVFcHBwfoIo0lkzG0j5lWrVhEZGVnr5xs2bGDSpElMmjSpwe1ER0eTm5uLjY0N0dHR1ebduHGDP//5zxw4cICBAyt7j1y8eDFdu3Zl/fr1ABw+fJglS5bwz3+a1IuYUiM0+Y00RWn4zZ28vDxiY2PJy8tj6tSpTJkypSVlbFVKA2/uNCZesIyYGxtrFXOKuUpTYv/pp59Yt24dAwcO5J///CcfffQRnp6erF+/Hh8fHyZPnszVq1eJiYnB09OT9957r1FlKC0tJTAwkOTkZNasWYO3t7cufQKVrRqEEPj7+zNw4ED2799PTk4OV65cYf369RQWFrJx40b+7//+r859NnReS8bTKle6PXv2ZO3ata2xaZMlY7Y8xshZ7969m0mTJvHNN98A8Ouvv/Ltt9+ycOFCkpIqh/yLiopCrVazbNky/QYsGYRJvQbs6empt22dOnWKOXPmsHr1ar1tszXoM+a0tDT8/f157rnnOHPmjN62qy+tGevdu3fx8vJi0aJFvPmmfkbxMUbOOj09nc8++4wDBw6QmJjIP/7xD3755RfWrl3LoUOHuHz5MqdPn2bjxo2cPHmS3bt3NzM6yVhafKWbmprK0aNHsbOzIzo6mq1bt6LRaOjZsyerV69mxowZDBkyhIyMDFxdXTl+/DgzZ87EycmJefPm4e7ujkajYevWrbptxsXFkZWVRWFhIZs3b2b58uXY2dkxcuRIPDw8GlWu4cOHExkZqbs60CdTjdnDwwMPDw9+/PFHDhw4wLBhw9pMrBMmTGDYsGG88soreHl5UVxc3OIHbcbIWb/xxhtAZasIT09PBg4cyJQpU8jOziYpKYkBAwawd+9eoPKP2KxZs1oUo2R4Lb7SzcnJwcnJiaVLlwJQUVFB586d+fTTTxFCUFFRQXh4OJMmTaJPnz4kJibyySefAODk5IRarcbBwYGzZ88CUFhYSFpaGt26daN9+/ZkZmaSn5+Pq6trtVu88vJyAgICqk0ajWGGJTPlmLVaLXFxcbpb2bYS6/Dhw/n1119ZtmwZ165dIy8vr8Wx9+rVC0dHR/Ly8pg3bx4A7733Hg8//LDugVZVTrWx+dwqr7zyClu2bNE9RFu5ciWDBg3SzQ8PD9c9RAOq7bNKU/cpmYYWV7ohISGMHTuWoKAgcnJyyMrKYv369fTo0QOtVkunTp2wsrLC1tYWOzs7bG1tKSkpAaCsrKzav1B5uzZgwADCw8OJi4tj9OjR7Nixg/Lycvz8/FpaXL0w1Zi1Wi1qtZqgoCAcHBzaVKxWVlZs2LCBzZs307VrV/r0aXn/3VU567feesssHhJK5qHF6YWEhASysrJQqVR0796d/Px8oqOjuXr1aoPrZmZmsnLlSm7duoWTkxMAXbp0YciQIQQHB1NaWkpoaCgRERGoVKpqVwLW1tbEx8fXuW2NRsPq1avJzMxk8ODBes0nmmrMmzZt4tSpU8THx/P0008zY8aMNhVrYGAgxcXFzJgxA2tr0+pWpCmtGxpy8uRJkpKS+Ne//sWCBQuYOtUURqmSGqtVmow1RlWOquYtk6Hpq8lYY5h6zJYYa01NiV2f+eyqSlcf+ewq+fn5REZGsmnTpkbHKRmf0Vov1JajsnRtKWZLiNWU89mpqak888wzPPvss4b5ZUh6Y1JNxiTJlJhqPhtg7ty5HDt2jLi4OP0FLBlEqyW+jhw5wsWLF1mwYIHetjls2DASEhKwt7cnPDyckpKSOvNkhw4dIi0tjfLyct58802uXbuGv78/SUlJ1Z4K61Nrxjx69GgCAwOxsbFhxIgRvPjiiw8su2TJEgoKCqioqGD79u1cvHix1WJuzVjv3LlDWloav/zyC4sXL2bChAkEBgbSrl07evXqVWsHMwUFBSxevBhra2tmzpzJ5MmTda8LN/eK21Tz2QcPHmTfvn3cuXOH2bNnNys2yYia2i0Z97rACwgIEL/99psoLCwUAQEB4syZM+LVV18V3t7eIjMzU3z11VciMTFRbN++XRw6dEgI8e8u79auXSuCg4NFUFCQKC8vF43VmB76q3h6eoqKigrx5ZdfiuTkZCFEZTd8WVlZ1ZajCV07GjPmo0ePioSEhAbjFqL6iAJNidlUYq2Sn58vAgMDRV5enq7rxFdeeUVcunTpgXV37NghDh48KISoPPZCCKHRaMSqVauaHHtL1bZfQ2vovJaTGXbtOH36dPbu3YuVlRVubm60b9+ekpIS7Ozs2LNnD2PHjq11vYyMDNLT0xk1ahRXrlwhNzeX/v37A/W/OtlUVW8S9evXj2PHjjVrGzUZM+aqLg0B2rVrV2cZ9TWigCkc34iICPz8/OjRowePPPIIS5cu5fLly+Tm5vLII49UWzY3N5fRo0cD1PoGmSFZQj5baj3NrnQnTpyIn58fQggSExNRq9WEhYWh0Wj47LPPdMvZ2tpSXl6OEIKioiIqKipwdnZuUh+kLVHzNcuWMGbM9vb2nD9/HqDOwQirRhTQR57P2Mc3PDycCRMmMHz4cACWL18OQGBgIH/4wx8eWL6qy8fBgwdXXblKkklqdqXbrl07unbtilarxdraGhcXFzZs2ICNjU211y/Hjx9PaGgo586dQ6vVMnToUIqLi3n55ZcpKCggNjZWt3x9r07e77fffiMkJITvvvuOrVu3EhwcTHR0dLWhXby8vFiwYAHl5eW1dj5tbjGPGzeO1NRUTp8+jaurK8ADMT/33HOMGTOGJUuWEBYWRt++fc0y1r/97W98/PHHXLt2jezsbPz9/Vm9ejU3btxg8ODB/P73v+fQoUP07t0bZ2dnAJ5//nmWLFnCBx98wAsvvNDsuBvDkPnsiRMnPrDstm3bePfdd/nggw8YOHAgFy5caPXnFZIeNTUfgRFHjK0vl/naa681uH5Lc7rGYMiYTTnWmiIjI0VBQUG9yzQ3p2tK+ey61DyuNX9u6LyWk/Ems2oyVlZWRnp6eq3zGupi8MKFC2RnZ5tdr/5tKeb6Yq0pJCSkwU5odu7c+UDutzGq8tl79+6tNZ9dl6p8drdu3SgqKiI3N1c3Lzk5uVq729TU1HrLUJXPliyPab0r2YBdu3Y1e93HHntMN8aUOWlLMbck1tqEhYU1az1Ty2dLlqXJla5KpbquKMrvWqMwxqBSqa43NN+S4oW6Y7bEWGtq6HiD6eWza+buU1NT2bdvHxcuXGDdunW1PliUTFeT+16QJEukz34nmqO+DnHCwsIaTCXd3/8uyL4XTJlZ5XQlyVK1pdx9WyevdCUJ41/p6pu80jVdZvUgTZJai6XlsxuTu5aMQ17pSlIzKIqSDCCEaPjpWMv2Mwg4BowTQmS25r4kw5A5XUlqIkVRpgMugLq19yWEOA+8BrynKIpta+9Pan3ySleSmkBRlD7Aj4C7EOJ/DbRPBdgHnBJCrDbEPqXWIytdSWqke5XffuAHQ1d+xqjspdYh0wuS1HgBQG/gr4besRDi2r39/z9FUboYev+S/sgrXUlqBFN5oGWoB3hS65GVriTVQ1GUGUAXIAjYLoR4y8jl6UJlmuFlwAuYaVENjNsA2U5Xkur3NDAAuAX83chlASgGVgHxVH5/+wAND9ommQyZ05Wk+o2ksnmYM5VXlsbWE4gCfgU6AI8ZtTRSk8lKV5Lq90fgAvCsECLcyGVBCHEVGARsB9oDzxi3RFJTyZyuJNXj3qvBv5hi3lRRlIeAQiFEqbHLIjWerHQlSZIMSKYXJEmSDEi2XpBMSocOHa4VFxdbVG9fRUVFfepbpi3G3JbJ9IJkUtpiv7ZtMea2TKYXJEmSDEhWupJFWLVqVZM+r0t0dDRqtbraQJAAOTk5eHl54e3tzalTp5pdTn1qizFbAlnpSmbnp59+Yu7cuYSFheHu7g7A5cuXyc7OxtXVlaioKJYuXar7vLFKS0vJzMxky5YtdOzYkUuXLunmvfvuu4SFhREfH8/bb7+t34AaoS3GbKnkgzTJ7CQlJRETE8NDDz2kq4CqODs7ExISgpdX7S+PJScnc+LECd3P48ePZ+7cuQDcvHmTXr16AeDg4EBubi6Ojo4A5Obm0q9fP1QqFcXFxa0RVr3aYsyWSl7pSmZHCIGiKFR2b1tdp06ddMs0VY8ePbhx4wZQeWttb2+vm2dvb8+VK1coKipCpVI1s+TN1xZjtlTySlcyO35+fixbtgxHR0c6d+7cpHV9fX3x9a29V0RbW1sGDx6MWq3GxsYGR0dHIiIicHd3x8fHh9DQUKysrAgODtZHGE3SFmO2VLLJmGRSGtN8Ki8vj9jYWPLy8pg6dSpTpkwxUOmaTl9Nxiwt5rZMVrqSSWmLbVbbYsxtmczpSm2Kp6en3rZ16tQp5syZw+rVpj1WpD5jTktLw9/fn+eee44zZ87obbtticzpSiYvNTWVo0ePYmdnR3R0NFu3bkWj0dCzZ09Wr17NjBkzGDJkCBkZGbi6unL8+HFmzpyJk5MT8+bNw93dHY1Gw9atW3XbjIuLIysri8LCQjZv3szy5cuxs7Nj5MiReHh4NKpcw4cPJzIykqSkpDYTs4eHBx4eHvz4448cOHCAYcOG6T12SyevdCWTl5OTg5OTk64dakVFBZ07d+bTTz9FCEFFRQXh4eFMmjSJPn36kJiYyCeffAKAk5MTarUaBwcHzp49C0BhYSFpaWl069aN9u3bk5mZSX5+Pq6urtWaY5WXlxMQEFBt0mg0bT5mrVZLXFycrtmZ1DSy0pVMXkhICGPHjiUoKIicnByysrJYv349PXr0QKvV0qlTJ6ysrLC1tcXOzg5bW1tKSkoAKCsrq/YvVDatGjBgAOHh4cTFxTF69Gh27NhBeXk5fn5+RomxJlONWavVolarCQoKwsHBQb9BtxEyvSCZvISEBLKyslCpVHTv3p38/Hyio6O5erXhocEyMzNZuXIlt27dwsnJCYAuXbowZMgQgoODKS0tJTQ0lIiICFQqFYMGDdKta21tTXx8fJ3b1mg0rF69mszMTAYPHqzX3Kmpxrxp0yZOnTpFfHw8Tz/9NDNmzGh5sG2MbL0gmRR9PsnPzs4mKSmJ9evX62V7zWHo1gvmEnNbJitdyaS0xeZTbTHmtkzmdCWzdOTIEb23Ghg2bBjp6elUVFSwcOFCFi1axPbt22tddtu2bfzpT3/i4sWLAFy4cAEXFxfdz62hNWP+8ssv8fPzY9q0aXzxxRe1Lnvo0CF8fX3x9vbmt99+M0jMlkhWupJJCgwMpKCggDt37hAYGMjZs2dZuXIl8+fP5/z587rlUlJSdJVEVU513bp1qNVqFi1ahFarbfQ+hw0bxpgxY/j2228ZOXIk27Zt48svv6x12UWLFuHm5qb7+bHHHsPFxaUZkf6bMWOeMGECiYmJ7Nixg48++qjWZXfu3ElSUhLe3t58+OGHeom5LZIP0iSTNH36dPbu3YuVlRVubm60b9+ekpIS7Ozs2LNnD2PHjq11vYyMDNLT0xk1ahRXrlwhNzeX/v37A/X3tnW/qt61ANq1a9cK0dXOmDFXiYiIqLM1Q1WHO/369ePYsWMtiLRtk5WuZJImTpyIn58fQggSExNRq9WEhYWh0Wj47LPPdMvZ2tpSXl6OEIKioiIqKipwdnYmPDy82fu2t7fXXVlWVFS0NJRGM2bMAOHh4UyYMIHhw4fXu1zN3sikppGVrmSS2rVrR9euXdFqtVhbW+Pi4sKGDRuwsbGp1s3g+PHjCQ0N5dy5c2i1WoYOHUpxcTEvv/wyBQUFxMbG6pavr7et+40bN47U1FROnz6Nq6srUDm6wv0jK6SmprJv3z4uXLjAunXr+MMf/mDWMf/tb3/j448/5tq1a2RnZ+Pv7/9AzF5eXixYsIDy8nJiY2NbHG+bJYSQk5xMZqo8JY3jhRdeqHPea6+91uD6a9asEVlZWdU+uxePjFlOukk+SJOke8rKykhPT6913tq1a+td98KFC2RnZ9OhQ4fWKFqraYsxG5tspyuZlLbYZrUtxtyWyZyuZFJUKtV1RVF+Z+xy6ItKpbremGXaWsxtmbzSlSRJMiCZ05UkSTIgWelKkiQZkKx0JUmSDEhWupIkSQYkK11JkiQDkpWuJEmSAclKV5IkyYBkpStJkmRAstKVJEkyIFnpSpIkGZCsdCVJkgxIVrqSJEkGJCtdSZIkA5KVriRJkgH9f7HCqNVwUH51AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "tree.plot_tree(clf) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也可以导出树"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "tree_pic = export_graphviz(clf, out_file=\"mytree.pdf\")\n",
    "with open('mytree.pdf') as f:\n",
    "    dot_graph = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\r\n",
       " -->\r\n",
       "<!-- Title: Tree Pages: 1 -->\r\n",
       "<svg width=\"550pt\" height=\"477pt\"\r\n",
       " viewBox=\"0.00 0.00 550.00 477.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 473)\">\r\n",
       "<title>Tree</title>\r\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-473 546,-473 546,4 -4,4\"/>\r\n",
       "<!-- 0 -->\r\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"377.5,-469 273.5,-469 273.5,-401 377.5,-401 377.5,-469\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-453.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">X[0] &lt;= 5.45</text>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-438.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.5</text>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-423.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 70</text>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-408.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [36, 34]</text>\r\n",
       "</g>\r\n",
       "<!-- 1 -->\r\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"316.5,-365 218.5,-365 218.5,-297 316.5,-297 316.5,-365\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-349.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">X[1] &lt;= 2.8</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-334.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.157</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-319.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 35</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-304.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [32, 3]</text>\r\n",
       "</g>\r\n",
       "<!-- 0&#45;&gt;1 -->\r\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&gt;1</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M306.669,-400.884C301.807,-392.332 296.508,-383.013 291.423,-374.072\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"294.421,-372.262 286.435,-365.299 288.335,-375.722 294.421,-372.262\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"279.806\" y=\"-385.704\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">True</text>\r\n",
       "</g>\r\n",
       "<!-- 10 -->\r\n",
       "<g id=\"node11\" class=\"node\"><title>10</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"432.5,-365 334.5,-365 334.5,-297 432.5,-297 432.5,-365\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-349.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">X[1] &lt;= 3.5</text>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-334.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.202</text>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-319.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 35</text>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-304.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [4, 31]</text>\r\n",
       "</g>\r\n",
       "<!-- 0&#45;&gt;10 -->\r\n",
       "<g id=\"edge10\" class=\"edge\"><title>0&#45;&gt;10</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M344.331,-400.884C349.193,-392.332 354.492,-383.013 359.577,-374.072\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"362.665,-375.722 364.565,-365.299 356.579,-372.262 362.665,-375.722\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"371.194\" y=\"-385.704\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">False</text>\r\n",
       "</g>\r\n",
       "<!-- 2 -->\r\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"200,-261 109,-261 109,-193 200,-193 200,-261\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-245.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">X[0] &lt;= 4.75</text>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-230.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.444</text>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-215.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 3</text>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-200.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [1, 2]</text>\r\n",
       "</g>\r\n",
       "<!-- 1&#45;&gt;2 -->\r\n",
       "<g id=\"edge2\" class=\"edge\"><title>1&#45;&gt;2</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M230.812,-296.884C220.648,-287.709 209.505,-277.65 198.95,-268.123\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"201.159,-265.402 191.39,-261.299 196.468,-270.598 201.159,-265.402\"/>\r\n",
       "</g>\r\n",
       "<!-- 5 -->\r\n",
       "<g id=\"node6\" class=\"node\"><title>5</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"316.5,-261 218.5,-261 218.5,-193 316.5,-193 316.5,-261\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-245.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">X[0] &lt;= 5.3</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-230.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.061</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-215.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 32</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-200.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [31, 1]</text>\r\n",
       "</g>\r\n",
       "<!-- 1&#45;&gt;5 -->\r\n",
       "<g id=\"edge5\" class=\"edge\"><title>1&#45;&gt;5</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M267.5,-296.884C267.5,-288.778 267.5,-279.982 267.5,-271.472\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"271,-271.299 267.5,-261.299 264,-271.299 271,-271.299\"/>\r\n",
       "</g>\r\n",
       "<!-- 3 -->\r\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"91,-149.5 0,-149.5 0,-96.5 91,-96.5 91,-149.5\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"45.5\" y=\"-134.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"45.5\" y=\"-119.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 1</text>\r\n",
       "<text text-anchor=\"middle\" x=\"45.5\" y=\"-104.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [1, 0]</text>\r\n",
       "</g>\r\n",
       "<!-- 2&#45;&gt;3 -->\r\n",
       "<g id=\"edge3\" class=\"edge\"><title>2&#45;&gt;3</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M119.111,-192.884C106.653,-181.226 92.6699,-168.141 80.2641,-156.532\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"82.4644,-153.797 72.7713,-149.52 77.6815,-158.908 82.4644,-153.797\"/>\r\n",
       "</g>\r\n",
       "<!-- 4 -->\r\n",
       "<g id=\"node5\" class=\"node\"><title>4</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"200,-149.5 109,-149.5 109,-96.5 200,-96.5 200,-149.5\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-134.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-119.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 2</text>\r\n",
       "<text text-anchor=\"middle\" x=\"154.5\" y=\"-104.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [0, 2]</text>\r\n",
       "</g>\r\n",
       "<!-- 2&#45;&gt;4 -->\r\n",
       "<g id=\"edge4\" class=\"edge\"><title>2&#45;&gt;4</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M154.5,-192.884C154.5,-182.326 154.5,-170.597 154.5,-159.854\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"158,-159.52 154.5,-149.52 151,-159.52 158,-159.52\"/>\r\n",
       "</g>\r\n",
       "<!-- 6 -->\r\n",
       "<g id=\"node7\" class=\"node\"><title>6</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"316.5,-149.5 218.5,-149.5 218.5,-96.5 316.5,-96.5 316.5,-149.5\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-134.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-119.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 29</text>\r\n",
       "<text text-anchor=\"middle\" x=\"267.5\" y=\"-104.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [29, 0]</text>\r\n",
       "</g>\r\n",
       "<!-- 5&#45;&gt;6 -->\r\n",
       "<g id=\"edge6\" class=\"edge\"><title>5&#45;&gt;6</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M267.5,-192.884C267.5,-182.326 267.5,-170.597 267.5,-159.854\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"271,-159.52 267.5,-149.52 264,-159.52 271,-159.52\"/>\r\n",
       "</g>\r\n",
       "<!-- 7 -->\r\n",
       "<g id=\"node8\" class=\"node\"><title>7</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"426,-157 335,-157 335,-89 426,-89 426,-157\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"380.5\" y=\"-141.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">X[1] &lt;= 3.2</text>\r\n",
       "<text text-anchor=\"middle\" x=\"380.5\" y=\"-126.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.444</text>\r\n",
       "<text text-anchor=\"middle\" x=\"380.5\" y=\"-111.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 3</text>\r\n",
       "<text text-anchor=\"middle\" x=\"380.5\" y=\"-96.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [2, 1]</text>\r\n",
       "</g>\r\n",
       "<!-- 5&#45;&gt;7 -->\r\n",
       "<g id=\"edge7\" class=\"edge\"><title>5&#45;&gt;7</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M304.188,-192.884C314.352,-183.709 325.495,-173.65 336.05,-164.123\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"338.532,-166.598 343.61,-157.299 333.841,-161.402 338.532,-166.598\"/>\r\n",
       "</g>\r\n",
       "<!-- 8 -->\r\n",
       "<g id=\"node9\" class=\"node\"><title>8</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"371,-53 280,-53 280,-0 371,-0 371,-53\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-37.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-22.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 1</text>\r\n",
       "<text text-anchor=\"middle\" x=\"325.5\" y=\"-7.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [0, 1]</text>\r\n",
       "</g>\r\n",
       "<!-- 7&#45;&gt;8 -->\r\n",
       "<g id=\"edge8\" class=\"edge\"><title>7&#45;&gt;8</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M361.264,-88.9485C356.206,-80.2579 350.736,-70.8608 345.633,-62.0917\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"348.534,-60.1189 340.479,-53.2367 342.484,-63.6401 348.534,-60.1189\"/>\r\n",
       "</g>\r\n",
       "<!-- 9 -->\r\n",
       "<g id=\"node10\" class=\"node\"><title>9</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"480,-53 389,-53 389,-0 480,-0 480,-53\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"434.5\" y=\"-37.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"434.5\" y=\"-22.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 2</text>\r\n",
       "<text text-anchor=\"middle\" x=\"434.5\" y=\"-7.8\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [2, 0]</text>\r\n",
       "</g>\r\n",
       "<!-- 7&#45;&gt;9 -->\r\n",
       "<g id=\"edge9\" class=\"edge\"><title>7&#45;&gt;9</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M399.387,-88.9485C404.353,-80.2579 409.722,-70.8608 414.733,-62.0917\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"417.871,-63.6557 419.793,-53.2367 411.793,-60.1826 417.871,-63.6557\"/>\r\n",
       "</g>\r\n",
       "<!-- 11 -->\r\n",
       "<g id=\"node12\" class=\"node\"><title>11</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"432.5,-253.5 334.5,-253.5 334.5,-200.5 432.5,-200.5 432.5,-253.5\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-238.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-223.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 31</text>\r\n",
       "<text text-anchor=\"middle\" x=\"383.5\" y=\"-208.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [0, 31]</text>\r\n",
       "</g>\r\n",
       "<!-- 10&#45;&gt;11 -->\r\n",
       "<g id=\"edge11\" class=\"edge\"><title>10&#45;&gt;11</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M383.5,-296.884C383.5,-286.326 383.5,-274.597 383.5,-263.854\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"387,-263.52 383.5,-253.52 380,-263.52 387,-263.52\"/>\r\n",
       "</g>\r\n",
       "<!-- 12 -->\r\n",
       "<g id=\"node13\" class=\"node\"><title>12</title>\r\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"542,-253.5 451,-253.5 451,-200.5 542,-200.5 542,-253.5\"/>\r\n",
       "<text text-anchor=\"middle\" x=\"496.5\" y=\"-238.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">gini = 0.0</text>\r\n",
       "<text text-anchor=\"middle\" x=\"496.5\" y=\"-223.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">samples = 4</text>\r\n",
       "<text text-anchor=\"middle\" x=\"496.5\" y=\"-208.3\" font-family=\"Times New Roman,serif\" font-size=\"14.00\">value = [4, 0]</text>\r\n",
       "</g>\r\n",
       "<!-- 10&#45;&gt;12 -->\r\n",
       "<g id=\"edge12\" class=\"edge\"><title>10&#45;&gt;12</title>\r\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M420.188,-296.884C433.103,-285.226 447.599,-272.141 460.46,-260.532\"/>\r\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"463.15,-262.819 468.228,-253.52 458.46,-257.622 463.15,-262.819\"/>\r\n",
       "</g>\r\n",
       "</g>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<graphviz.files.Source at 0x1e598821cd0>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graphviz.Source(dot_graph)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者，还可以使用函数 export_text以文本格式导出树。此方法不需要安装外部库，而且更紧凑："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import export_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:70: FutureWarning: Pass feature_names=['sepal length (cm)', 'sepal width (cm)'] as keyword args. From version 1.0 (renaming of 0.25) passing these as positional arguments will result in an error\n",
      "  warnings.warn(f\"Pass {args_msg} as keyword args. From version \"\n"
     ]
    }
   ],
   "source": [
    "r = export_text(clf,feature_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|--- sepal width (cm) <= 3.15\n",
      "|   |--- sepal length (cm) <= 4.95\n",
      "|   |   |--- sepal width (cm) <= 2.65\n",
      "|   |   |   |--- class: 1.0\n",
      "|   |   |--- sepal width (cm) >  2.65\n",
      "|   |   |   |--- class: 0.0\n",
      "|   |--- sepal length (cm) >  4.95\n",
      "|   |   |--- class: 1.0\n",
      "|--- sepal width (cm) >  3.15\n",
      "|   |--- sepal length (cm) <= 5.85\n",
      "|   |   |--- class: 0.0\n",
      "|   |--- sepal length (cm) >  5.85\n",
      "|   |   |--- class: 1.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(r)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 决策树回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a random dataset\n",
    "rng = np.random.RandomState(1)\n",
    "X = np.sort(5 * rng.rand(80, 1), axis=0)\n",
    "y = np.sin(X).ravel()\n",
    "y[::5] += 3 * (0.5 - rng.rand(16))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABHdElEQVR4nO2deXhU1dnAf+9kgZCwMyCLECoqcUF2tCiKu9Yd+wnaFimySLVWv9Zal09t1dpW2yJiBUSpG2JVqrZarRXFpZQdRKCCGmRnCBCSkD3n++Pemcwkk1mSWe5M3t/zzJOZuWfuPXcm977n3cUYg6IoiqI0hSvZE1AURVGcjQoKRVEUJSQqKBRFUZSQqKBQFEVRQqKCQlEURQmJCgpFURQlJCooFMcjIm+LyMQIxpWKyLcSMSelaUTkDBH5b7LnocQO0TwKJRaISCHQA6gBaoGNwLPAXGNMXRKn1iJEpNTvZTugEuv8AKYZY16I03ELsb7PWqAU+AdwkzGmNNTnFCUeqEahxJJLjTHtgX7Aw8DPgfnJnVLLMMbkeR/AN1jn6H3PJyREJDMOh7/UPu5gYAjwi1gfIE7zVtIMFRRKzDHGFBtj3gCuASaKyEkAItJGRB4RkW9EZK+IPCkiOd7PicjlIrJWRA6LyJcicqH9/gcicoP9fICIfCgixSKyX0QW+X3eiMgA+3lHEXlWRDwisk1E7hYRl73tehH52J7LQRH5WkQuiuYcReQsEdkhIj8XkT3AMyLiEpE77LkXicjLItLF7zOnisinInJIRNaJyFkRfp97gHewBEbYfYlIfxFZKiIlIvKeiMwWkeftbfn29zRZRL4B3rff/6GIbLK/j3dEpJ/9vojIH0Rkn/2dr/f7PS8WkY32cXaKyE/9vxu/+RTYv+EhEflcRC7z27bAnt/f7f38R0SOiea3UOKPCgolbhhjlgM7gDPst34DHId1wxsA9Ab+D0BERmKZqn4GdALGAIVBdvsr4F2gM9AHmNXE4WcBHYFvAWcCPwAm+W0fBfwX6Ab8FpgvIhLlKR4FdMHSoKYCPwausI/XCzgIzLbPrzfwd+AB+zM/BV4VEXe4g4hIH+AiYGuE+3oRWA50Be4Dvh9kt2cCBcAFInIFcCdwFeAGPgIW2uPOx/otjsP6Xa4Biuxt87HMb+2Bk7CFToO5ZwFvYv1m3YGbgRdE5Hi/YROA+7F+063Ag+G+EyXBGGP0oY8WP7Bu6ucGeX8ZcBcgQBlwjN+204Cv7edzgD80se8PgBvs588Cc4E+QcYZLAGUgeVLOMFv2zTgA/v59cBWv23t7M8eFek5AmcBVUBbv+2bgHP8XvcEqoFMLDPccw329w4wMcSxSoESe27/AjrZ25rcF9AXy0/Uzm/b88Dz9vN8e3/f8tv+NjDZ77ULOIIlAM8GvgBOBVwNjvmN/b12aPD+WcAO+/kZwB7/z2IJofvs5wuAp/y2XQxsTvb/sz4CH6pRKPGmN3AAa6XaDlhlmyAOYTlovavgo4EvI9jf7VhCZ7ltxvhhkDHdgGxgm9972+y5eNnjfWKMOWI/zYvg+P54jDEVfq/7AYv9zm8TljO6h73tu95t9vbTsYRJU1xhrNX6WcBA+7y8x2lqX72AA37nBLA9yL793+sHzPTb1wGs77i3MeZ94HEszWiviMwVkQ7258Zh3di32ebA04Icpxew3QQGNDT5W2AJqGh/ByXOqKBQ4oaIjMC6IXwM7AfKgRONMZ3sR0djOWvBunGFtU0bY/YYY6YYY3phrWaf8Pol/NiPtZLv5/deX2Bny86o8XQavN4OXOR3fp2MMW2NMTvtbc812JZrjHk47EGM+RBr5f2I33Ga2tduoIuItPPbxdFh5r4dy4Tkv78cY8yn9vEfM8YMA07EMkH9zH5/hTHmciyT0l+Bl4McZxdwtNc/ZBOP30KJIyoolJgjIh1E5BLgJSyTx2f2inIe8AcR6W6P6y0iF9gfmw9MEpFzbKdwbxEZGGTf37Vt9mD5AAz14aoAGGNqsW5aD4pIe9sxexuWCSaePGkf0+sIdovI5fa254FLReQCEckQkba207dPk3sL5I/AeSIyONS+jDHbgJXAfSKSba/yL41g3r8QkRPteXcUke/az0eIyCjb11AGVAC19r6vE5GOxphq4DANfgeb/9ifu11Esmyn+6VY/xtKiqCCQoklb4pICdYK9S7g9wQ6kH+O5axcJiKHgfeA48Hn+J4E/AEoBj4kUCPwMgL4j1j5DW8Atxhjvg4y7masG9RXWBrNi8DTLT3BMMy05/Su/T0sw3KaY4zZDlyO5TT2YH1HPyPCa9AY48Hyz9wTwb6uw/L/FGE5vBdh+Wya2vdirECDl+zfZQOW8xygA5aAP4hlMiqiXrP5PlBof2Y68L0g+64CLrP3tx94AviBMWZzJOetOANNuFOUNEesEOLNxph7kz0XJTVRjUJR0gzbXHSMbcK7EEv7+GuSp6WkMJqVqSjpx1HAa1h5FDuAG40xa5I7JSWVUdOToiiKEhI1PSmKoighSUvTU7du3Ux+fn6yp6EoipIyrFq1ar8xJmhJmbQUFPn5+axcuTLZ01AURUkZRGRbU9vU9KQoiqKERAWFoiiKEhIVFIqiKEpI0tJHoShKcqiurmbHjh1UVFSEH6wkhbZt29KnTx+ysrIi/owKCkVRYsaOHTto3749+fn5RN8HSok3xhiKiorYsWMH/fv3j/hzSTU9icjTdovFDU1sP8tuv7jWfvxfoueYbng8HlasWIHH40n2VJQ0pKKigq5du6qQcCgiQteuXaPW+JLto1gAXBhmzEfGmMH245cJmFPasmjhQgYO6Mf08ecxcEA/Fr20MPyHFCVKVEg4m+b8Pkk1PRljlopIfjLn0FrweDzMmD6ZJTeUM6hXOet3wdhpkzn7nHNxu8O2bVYUpRWTbI0iEk4TkXUi8ra3sUowRGSqiKwUkZVqVmlMYWEh+d2yGdTLej2oF/TrmkVhYWFS56UoivNxuqBYDfQzxpwCzCJEqWRjzFxjzHBjzHBdITcmPz+fwv1VrN9lvV6/C7YVVaOlThQlNuTn57N///5mfXbBggXs2rWrxfvavn07Y8eOpaCggBNPPJGZM2c2az4NcbSgMMYcNsaU2s/fArJEpFuYjylBcLvdPDFnPmOfymHo7A6MfSqHJ+bMV7OTojiAhoKiuWRmZvLoo4+yadMmli1bxuzZs9m4cWPL99viPcQRETkK2GuMMSIyEkuwFSV5WinLNeMncPY551pmqPx8FRJKXJnyxIG47HfejC4htxcWFnLhhRdy+umns2zZMk455RQmTZrEvffey759+3jhhRcA+MlPfkJ5eTk5OTk888wzHH/88fz+979nw4YNPP3003z22WdMmDCB5cuX065du0bHKSoqYsKECXg8HkaOHIl/y4bnn3+exx57jKqqKkaNGsUTTzxBRkYGeXl5TJs2jSVLltC5c2deeuklPvzwQ1auXMl1111HTk4O//73vwGYNWsWb775JtXV1fzlL39h4MBGLeQb0bNnT3r27AlA+/btKSgoYOfOnZxwwgkRf7/BSHZ47ELg38DxIrJDRCaLyHQRmW4PuRrYICLrgMeA8UYbaLQIt9vNiBEjVEgoac3WrVu55ZZbWL9+PZs3b+bFF1/k448/5pFHHuGhhx5i4MCBLF26lDVr1vDLX/6SO++8E7CEx9atW1m8eDGTJk1izpw5QYUEwP3338/pp5/OmjVruOyyy/jmm28A2LRpE4sWLeKTTz5h7dq1ZGRk+IRTWVkZQ4cOZfXq1Zx55pncf//9XH311QwfPpwXXniBtWvXkpOTA0C3bt1YvXo1N954I488YrUpX7JkCYMHD270+Pa3v91ofoWFhaxZs4ZRo0a1+PtMdtTThDDbHwceT9B0FEWJIeFW/vGkf//+nHzyyQCceOKJnHPOOYgIJ598MoWFhRQXFzNx4kS2bNmCiFBdXQ2Ay+ViwYIFDBo0iGnTpjF69Ogmj7F06VJee+01AL7zne/QuXNnAP71r3+xatUqRowYAUB5eTndu3f37f+aa64B4Hvf+x5XXXVVk/v3bhs2bJjvOGPHjmXt2rVhz7+0tJRx48bxxz/+kQ4dOoQdHw5Hm54URWk+Ho+n1ZoZ27Rp43vucrl8r10uFzU1Ndxzzz2MHTuWxYsXU1hYyFlnneUbv2XLFvLy8iLyGQTLSTDGMHHiRH7961836/MNzyEjI4OamhrA0ihuvfXWRmPbtWvHp59+ClhlVMaNG8d1110XUhBFg6Od2YqiNA9NrgxNcXExvXv3BixHsv/7t9xyC0uXLqWoqIhXXnmlyX2MGTPGZ1J6++23OXjwIADnnHMOr7zyCvv27QPgwIEDbNtmtXqoq6vz7fPFF1/k9NNPByx/QklJSdh5ezWKhg+vkDDGMHnyZAoKCrjtttui+UpCohqFoqQZHo+Hea/eyp9e6EX7NlBSCct33EfbjUvIzs4O+hkhg0E9vk+v9sMTPNvkcPvttzNx4kR+//vfc/bZZ/vev/XWW5kxYwbHHXcc8+fPZ+zYsYwZM8ZnOvLn3nvvZcKECQwdOpQzzzyTvn37AnDCCSfwwAMPcP7551NXV0dWVhazZ8+mX79+5Obm8vnnnzNs2DA6duzIokWLALj++uuZPn16gDO7OXzyySc899xznHzyyQwePBiAhx56iIsvvrjZ+wSQdPQNDx8+3GiHO6W1smLFCj4qmkz77pFXBwXo23EMFxzzhxYde9OmTRQUFLRoH+lMXl4epaWlyZ5G0N9JRFYZY4KuFFSjUJQ0Iz8/n6WHLNv32I1fs2tPLT//Rzbznn6Wjh07NhrvOfI5q3Y/SZ2pSfRUlRRBBYWipBlut5uOHdtjKGfio7D5q2qemPMkJ/W9KMwn08+6EAueeeaZRhnOo0ePZvbs2VHvywnaRHNQQaEoaUh2m2wqa8t5ZO6rDMg/KWTUkzfyxpi6RE0vpZg0aRKTJk1K9jSSigoKRUlDDLUADB0yjDaZ4eLobUGhGoXSBBoeqyhpiFc7EAl/iYvvNqCCQgmOCgrFMWj3vdhRZyyNQsgIO1ZNT0o4VFAojkATxGKLIXKNwmt6UpSmUEGhJB3/7nurZhSz5IZyZkybrJpFC/BqB64oTE9e4aI0Dyf0o/B+1ptwN3x4bBIo1ZmtJJ367nvlQGD3vdZWoyhWeJ3ZanpKDRYsWMBJJ51Er169YrK/JUuW0K1b7Fr3qKBQko5/971BvbT7Xkvxv+En0/Q0b/WwuOx3ytBVIbe35n4U8UJNT0rS0e57sSUabcIal36mp9bcj0JEOP/88xk2bBhz586NyfepGoXiCLT7XuyoiyI01h5o/Y1x3bdwK/940pr7UXzyySf06tWLffv2cd555zFw4EDGjBkT8jPhUEGRoqRjrwG3250255JMfDkUERoMJA0T7lpzPwqvn6N79+5ceeWVLF++vMWCQk1PKYiGkiqh8JqeXNJ6TU/hSNd+FGVlZb79lJWV8e6773LSSSdF/L00hWoUKYZ/KOmgXuWs3wVjp03m7HPO1dW4Avgl20UoKOJlenIy6dqPYu/evVx55ZUA1NTUcO2113LhhRc2e39etB9FirFixQqmjz+PVTOKfe8Nnd2BOYve89lEldZNefVBnv/sXNpkdOQHp7wfdvz+I5tZvPk6uuYcz1UFL7bo2NqPIjSp2o9CTU8phn8oKWgoqdKY6E1P6eejUGKLmp5SDF8o6bTJ9Ouaxbaiag0lVQKIpiCghT1OE+6Cov0okiwoRORp4BJgnzGmkcdFrJCAmcDFwBHgemPM6sTO0nmkYihpdW05xZXbAt47ePAgu3bt8kVpeJ97wwybS7usbrTLil1WaqoRdR6FqEYRCu1HkXyNYgHwOPBsE9svAo61H6OAP9l/Wz2pFkq6ePN1jQQFABnwxd4Gz/c2HhYNQgb/c+JiOrTp3bIdpShR51Go6UkJQ1IFhTFmqYjkhxhyOfCssTzuy0Skk4j0NMbsTswMlWhpKr+juGIbCOz84ggZLqiphePckJUBG/fCsd0gJwvKq2FrkYuCE04kMzPw3/PgwYNs/2YbBkOWC2rqhKP79gvQQEqqdlJVW8rB8q2tVlBEn0ehpiclNMnWKMLRG9ju93qH/V4jQSEiU4GpgC9MTUksixYuZMb0yeR3y6ZwfxVPzJnPNeMnWPHkdgTmlSs2U1UDU16BtbfBim/gN6/CKr8coqGzO3DxogUBUVwej4fjRvZFaiv44Mb6mlBjpxay+cttPqH04bZf8kXR65TXFCXy1B2Fz/QUqTNbTU9KGJwe9RQsbTHof7MxZq4xZrgxZngqmWTShVClwgu3fQ2AqTOc0gvyu8D2Q9aNPr8LfH2AsFFchYWF9OiQQf8ulpAA62+fzi4KCwt943IyLe3iSPWBOJ+xc/GVGI/48k5fQXHffff56iQF469//SsbN25M4IxSE6cLih3A0X6v+wDh8+qVhFNfKtx67V8qvF+/fgDU1VmCwJ0HPx8Lp86C8xe0p5osxszJDlkQMD8/n72HaxsJlR0H6wKESk5WVwAqalqxoIhWo/DdBlqf6UkFRWQ43fT0BnCTiLyE5cQuVv+EMwlVKrxL106ww6qBc9rj0LM97K/IZubjjzF4yFDfjT5UFJfb7ebJeU9zww8nctrj1b59zHkqUKh4NYryJjSKTZs2sXz5ckaOHJm2iWHRhsfWJ2YnR6OIdd2yBx98kGeffZajjz4at9vNsGHDmDdvHnPnzqWqqooBAwbw3HPPsXbtWt544w0+/PBDHnjgAV599VXef//9RuOaqh7bmkh2eOxC4Cygm4jsAO4FsgCMMU8Cb2GFxm7FCo9t3TFqDiZUfkdNXSUA2VltWPzmOwAMGTKk0U0h3E3CGxa8Zs2aJveRk9kFgOLKb9hVsiJg2+OPzeJvby6mex48+ic494zr+P3vnmz+STuUOqJzZuOr9ZR4QdGUX6u5rFq1ipdeeok1a9ZQU1PD0KFDGTZsGFdddRVTpkwB4O6772b+/PncfPPNXHbZZVxyySVcffXVAHTq1CnouNZOsqOeQv5H2NFOP0rQdFo9LV3ZNZ3fUb/CPf/881s0R7fbHXIfXtNTUfl/+fuW6QHb+l8EN190nO/1wb2fsHHjRtxud0rlpITDRFnrKVmmp3jULfvoo4+48sorfVrAZZddBsCGDRu4++67OXToEKWlpVxwwQVBPx/puNaG030USoKIVUVat9vNiBEjAi50r0kj8rj+5tO57TGc0O279MwbFvCQ0t58s66EngetB0DnHtnMnvUYAwf0Y+JVY8k/uhc/ueWWlO/VHa0zO1mmp1B+rZYQrHT39ddfz+OPP85nn33GvffeS0VFRdDPRjqutaGCQmHTpk1MnzopaMRSLDBRm0Kaj4gwuu8dXHLc3IDHaPdveHTaFvq+tYVL1m6BOuumuOilP3PhMeV8tbsM6mr42wuP8a2+PVO6dHu0zuz620BiBUU86paNGTOGxYsXU15eTklJCW+++SYAJSUl9OzZk+rqal9pcGhc3rupca0dFRStGI/Hw0MPPsCpw4fQPacy5is7L/UaRXx6M0dCQUEBU6bfxKmz4LiHoda2shzVMYPXPoN22fDvm2HrL+CTGbXcOPWHKatZRO3MTlJ4bDxa4A4dOpRrrrmGwYMHM27cOM444wwAfvWrXzFq1Chfxzcv48eP53e/+x1Dhgzhyy+/bHJca8fpUU9KnFi0cCHTp/6QqqoK/v5DGPcsQSOWYkEiNYpQzHxsFtNvnMHy5cupzpyNoZbtB6rpngvd8gLzM3p3FAoLC1PSZ1Hnq/UUqenJFhRJyMyOR92yu+66i7vuuqvR+zfeeGOj90aPHh0QHnvjjTcGHdfaUY2iFeJ1Is6+rIKBbjhrADxxFYx90lptnzm3TYwr0npvQMnTKLwUFBQwceJEXwnue++/n31ljZP+dhablC3d7nVmuyTSdWByTE9egvm1FGehGoVDqKg5xEbPy1TXHon7sXbv2cP4W3vS84RKCnrB33pA/jh48mz468YMrvv+D+jaZS//2fHHZh9DxMWALhfTJWdAQp3ZkWMJrZt/fAubN27hz888zWmzoGcH2Hckk3lPP93oxpUqfcpTxfSkpA4qKBzCF0VvsGr3nMQczAWDL+vEVuDMAVbhrN0AfeGsU2BnzVvs3Nfywxwo38qFAx5zjOnJHxGxF9CGufPmc+ttP+W9996jR48ejB07tpEgiHW8fzxJtunJGJNUf5QSmuZEt6mgcAjVteUA9G4/it4dTo3JPstKSzl48CCdO3cmNy8vYNu6tWtY/Opf6NwugwNltZw59mxGjTq10bjmUFK5g037X6W6zjonJzizGyK+JDPr5lhQUNBkpnaq9Smv1ygSH/XUtm1bioqK6Nq1q6N+b8XCGENRURFt27aN6nMqKByC94Z1VN4QTunxgxbty+PxMG/uHB79zUPku4OvgE+54AdcNPTOuJhSdpeuYdP+V322cidqFL5CeBGsrurj/S3B5x8V5mhBEXGZ8diZnvr06cOOHTtSNmKsNdC2bVv69OkT1WdUUDiEaC/upvCPZvr3TYRcAcer+VHjG49znNle6u334W+OoepYOZGoe2bH0PSUlZVF//79W7wfxVk4aYnXqvGtulugrjeMZopXXkQ4vCYPn0bhQGd2vTALf3OMR7x/PIl+0eEcAa44E9UoHEJVjXVxb9hWy9avmhf5tHvPYUZfdQ81J1aSVw1/qrBKentKof2ILDYd7s+2T+MfVVUjldAGDpTW8sqnR6iVI9AGjlTAKwk4fiRUtREQeGN5GS6yw47P6Hs5f3jtPIqLi+nYsSMZOe0ccy4NqXSVQzZsLzK8sif8HA0V0La+haqiNEQFhUPYUVQFwH931bFnd3Pry3Smx4hpfAQcdxasrn+b486Ej/4LEP/aNe1yazjpZDhcXsO/P6ugTZtyThkCR6qEZWudUTtnyDAhKwuWfF5BTXWkc3IBnWEPJOJ7bC5du1VyzADYfcDw1Zfh5+lyVTF8JNTUqaBQgqOCwiHU1NWBC3p0zGB0fk5En3njr6+z/D+f0jEHisth5Knfpl+/fix+7RU7mqmGM8/yRjPlxvkM6qkyOewFOuXCuNNyqDbZ7AHy2roYd1pk5xZvdhqhDrhkWBsyxBlzihWlJoODQF93NkO6hz+33QcNNUCyEu4U56OCwiF47fjuDllcOCT0xb1p0yYWL17Mgl/fxbKb6x2spz40i1XrNvLDS+5IamJY0ZF2vLYZ2reFC0/I4VBFNn/ZCLltMrjwRGfclJ9f76K8Bs46qQ252c6YU6zYvD+Dj76BPl2zObNf+HNbv62W/xSBCgqlKVRQOARfpIortAPyJzffzNw5j9OhDfTp2KB/dEdYvnw5EydOTKqj1euQ9zbQcaQzWwLzKOJJojO6oy0z7ooiAkxpnTjnym3leG+mriainjweDy+//DJz5zzOspthyY2wo7hB/+hiGDlyZKKm3CS+aBv7huXEPApJUH2jWPX5iIb6MuORCgrv/5z6KJTgqEbhEOpDGhvHvnvLR+RlmwAtYsooOHWWpUnsKIYp029yRB9o7w2qkUbhwDDMeDbrSVZGd12UmdnR5JQorRPnLPFaOd5Vd0PTk//N5h+TKgK0iMkjoa4Orr/1QVat28jMx2YletpBadxa0/7bykxP8ergFo5oNTivABdRQaEERzUKh1CvUQSuuhuWj2ioRUybcRN33nlnwucbCp9G0cj05ByNIhF9opOV0R1tz2yXq/530YJ+SjBUUDgEb7kLV4NVd8ObzeSRsGBNG27+v99x7rnnOsLU1JCGN2FnOrO9ZSvidwxfRve0yfTrmsW2ouqEZHRH78wGYwQRg6EuqPlTad2ooHAIvovbFXiRBrvZzH3KuSWuwc+s42BnNlGU8GgJ4Tq4xSMiqi7KntmWzBQsH4Wan5TGqKBwCL5IlSDmmXi0i4wnDUt4e7Ulp5qeqmpL+HT7I1TUHIzfATtDUTFQXP/W7l272LDhM9plC0dWGE46+WR69uzV4kMVV24HojA9iWCMC5E6S/tzzs+kOISkCgoRuRCYCWQATxljHm6w/SzgdeBr+63XjDG/TOQcE0VtbQ0ZQFVlVdDt8ar0Gg/qNQp7dWoc6Mz2KzO+8/Bythz4W+InkQcDT63v/1HD12w//HWID0RHblb3iMbVaxSgGoUSjKQJCrGWO7OB84AdwAoRecMYs7HB0I+MMZckfIIJZNHChezK2MnRA+C5Pz9D5ZZaR5uWwlGvUTTsR+Ggpaov6slQayzhfFTeUE7pMTEhh9/yxRc8/pu7mfmdMt97P/5bO26+40GOPe64Fu8/y9WOHnmnRDTWX1AkIgFRST2SqVGMBLYaY74CEJGXgMuBhoIirfF4PMyYNpn7F10GwHdPqGKGg7unRUK9o9gE/HWUM5v6JDPv/PKyetC34+kJOX5O/vEs/edNHOpX7ouI+ui9auY9eSnujon93UXAmAZaoKL4kcwrtzew3e/1Dvu9hpwmIutE5G0RObGpnYnIVBFZKSIrU6W7lsfj4d7/u4cubcrJtM3J3XPrEto7Ih54o2YaaxTOExTGmPpVdAIFWageFx6PhxUrViSsS1zgWaugUBqTTI0imB2i4X/paqCfMaZURC4G/gocG2xnxpi5wFyA4cOHO/6/3duJrrKygraZUF1nOWr2l9axbb9zu6dFQiONwoEd7vDLRvYlOyZYkAULUvBm4ed3y+arfZX87I67mDJ1Wly1SxHAxD8BUUldkrnE2wEc7fe6D7DLf4Ax5rAxptR+/haQJSLdEjfF+ODfia6gO/xpHByqsG6iTy+r5X9/fmfKmp2gsUaBg01Phrr6FqBJmJ/b7WbEiBE+TcKbhX/7t4uR2gqe+eM9DDwmvjWiRMRvheb4NZaSBJJ55a4AjhWR/iKSDYwH3vAfICJHib08FZGRWPMtSvhMY8yaNWs4urOL846DwoNQ0B265HpDSDOYMnVakmfYMprSKJzkzPY3PZEkjaIh3iz8nh1gxmvwwY2w5Q5YMqWcGdMmx80UZf1c6qNQmiZpV4Yxpga4CXgH2AS8bIz5XESmi8h0e9jVwAYRWQc8Bow3Kf6fvGjhQq4Zdzlbdpex+zA8cRWc9ScorrS2X37FlSmtTUAQH4UDNQr8aj3VOSR815uF/88vIL9z4nqeuwQwGvWkNE1S8yhsc9JbDd570u/548DjiZ5Xc6ipK2f17qdCJm1VVFTwr62LeGZ+dw6Vw6xCaNMFxt2RgbvXFgCGDB6SoBnHj6Y1CucICgnIG7DmV1leyYoVK5KW1Oh1cE+f8kOqqioSViNKBIzmUSgh0MzsGPFN8Ses27sg7LgR3+mE14Aw8uTG2xuW8EhFmop6cpIzOyDqyRZoLzz3LOsXzqRwfxVPzElOmRSvg3ve3DmM/c1D9OsW/xpRanpSwqGCIkZU11qJUz1yT+G4rpcGHVNSUsI9d97BT8+ook8n2HEIHvkom0l3juZw9VdA46KAqUlgNVInOrP9TU8lpYcBuPKEGp6ZUZywvhFN4Xa7ufOuu5kydVrQsi2xrg/l8pZ5or7ciqL4o4IiRtQay8nQJedYBna7MvigbjDp8iyuCagmOosu7VZxuNgWFGFaoaYCIoLgsiKK7Ac405kNhqID+8EFbjugwN8nkEx/UbCyLf7hs7HSfCw5Ef+y60rqooIiRtTUWWUgMlxtQo4LFjv/3ldrfNsznLTqbhFe00595rOjfBR+9ag6d+7MrmLYX2JtS1TfiGiJV8c8y6cU6FdSFH9UUMQIb72gDMkKO7bhStEl9T9DOmgUYN2IjakN1CgcJQTro3xyc3OgGBatz+TeVzokrG9EtDRsYhUrzUcCop5UUCiNUUERI+oi1CiC4V8OOj18FH6FAU2ds53ZfoLshik3kvvdsx1byj1eHfNcAVFPanpSGqOCIkbU2BrF7p378GR6orrRuPw6imWkkUaBsSOeHOjM9s3F1K+ic9vlMeLYEUmcVWhCdcxriYNbo56UcDjnyk1xNm/+DIBX/jyPgQOiK7ngfwNNFx9FMI3CSc5sf9NTfY9pJ80vONeMn8DmrduYs+g9Nm/dxjXjJ7Bo4UIGDujH9PHnRf2/Bw0T7lRQKI1Jj7tSkvF4PHz08b8AuP3bR1hyQ3QlFyRNfRTgvRE70JkdYHpy3vxC0VR9qFUziqP+34OGCXdqelIakxpXhsMpLCykU3vLfJRZVxd1yQVX2vso7OKADjq3etOT8XXgc5JpLFLqHdzWa+//3po1ayIuVR4Y9RTHySopS+pdGQ7C2zcgLy+PijrrCnPVmaidjP4+isx01Cgc3DPbKjLuXUWn3nfv7+AGy8G9dU85E66+ImJTVGDUk2oUSmPUmd1MGiY+3fnn0cABfv73tnzwdk5U4ZX+K9l0KOEB/mac+jLeTlyxW852584vHA0d3F97qnC56lgyJfJcC024U8KhgqIZBEt8enbPN3Tom8eM/32QJ2ddElXkib+PIiMFHKqR4L3pWpVZnVgUsN705MSihdHgn8R58OBBfnHj/zColxWFF0muhcvvX05NT0owVFA0g2CJT3ntrJtMwcCTcLePLjzR3/SUnhqFc8NjDcbRGk+keJM4PR5P1LkWAT2zVaNQgqCCIgq8sep5eXmNLsaa7taYDMmOer/+zuzMNNQoHB8e68j5NY9QuRZNYbcGs1+pSqE0RgVFhDT0SXxv4mTGPjXfdzH+9u186thHhit6QeHvRJV0cWb7axQODD/1CQVj6ucn6aHNBasnFoqAPAq1PSlBUEERAZs2bWL61El8OLWy3kH41Hw+XraK0tJS8vPz+WDfdA5VQoZEX8ID0jA81l+jcEgHOX8C8zzshLs00Ci8BKs82xRWz2yvoKiN57SUFCXslSsi/SN5L12ZN2cOI4edQvecykax6qWlpb7Ep1pTDUCGK3xRwIaIn6Dwf57KSEAUjRNNO/WZyE7UeLyh1/Hqk90Y6/uoU41CCUIkV8arQd57JdYTcSLz5szhlpun8+bEavaXERCr3tBB6O1H0VKNIhXKSERCoEbhXGc2xnnO7JaW5Gge3t9LBYXSmCZNTyIyEDgR6CgiV/lt6gC0jffEks3n29/htY/v56qJ3XCfCX88Bn6xFvIOQkl1Fn986XvsYwn77AVfVW0pQMt9FA5a1baEQB+F8zSKYNVjnfDdx6vnRFg04U4JQSgfxfHAJUAnwL+3ZwkwJY5zSjpHqj18uu8urrq1JwCfAhwHV5xdP6aCf/Hp9n8FfE7IINPVDBlq/DWK5N+sYkF9YyBnOrP9o3zqNYrkC7J49ZwIj5qelKZpUlAYY14HXheR04wx/07gnJLC7oO1vLjU6nvtytxNXk9DVWVbSnafREklZAjU1EGnzp3IbZcbdB81VScy880qoCqqY2e0qyG3q/XcWTfT5lNfIsPhzmz/qCcH+Ifi1XMiPCoolKaJJOqpSET+BfQwxpwkIoOAy4wxD7T04CJyITATy0j/lDHm4Qbbxd5+MXAEuN4Ys7qlxw1GRZVh884aAHJyqjm5J9TUHsWXe34dMM5zKNyeaqI+ttsN/b2CwgGr2ljgM+0Yg7Od2c4qM96cPIjYoOGxStNEIijmAT8D5gAYY9aLyItAiwSFWEHrs4HzgB3AChF5wxiz0W/YRcCx9mMU8Cf7b8w5qnMGt13WHoCymhzWHIBu7TP44Zha9u7dS48ePejUqVM8Ds3e8ly2lHhfOWfV3RK8OQmGWoeXGa/XKJzy3UebBxEb/EuuKEogkQiKdsaY5Q1WW9EvmxszEthqjPkKQEReAi4H/AXF5cCzxrrTLBORTiLS0xizOwbHDyAnWyjoY4W2Fh3JYM0BaJudwWkFbjgpvhdqZlEbn6BImzwKvxVqfc/s5K/YvQQrM+6k7z6aPIjYYP9edSoolMZEcmXsF5FjsHP7ReRqIBY36t7Adr/XO+z3oh0TcxIdpeMKyAh2zs2qJQRoFA50ZgeLekqX77552D4KLeGhBCESjeJHwFxgoIjsBL4GvheDYwe7Czf8L41kjDVQZCowFaBv374tmlii4/79j+Okm2nL8LN5O9CZHZBwZ5wTHps8vBqFCgqlMWEFhW0aOldEcgGXMaYk3GciZAdwtN/rPsCuZozxznMulkBj+PDhLfpvT3RcvX+NoXQJj3UFaBTOc2YHJNzhrIS75KA9s5WmCSsoROS2Bq8BioFVxpi1LTj2CuBYuxzITmA8cG2DMW8AN9n+i1FAcTz8Ew1JdEc2l9/PkD6rWj8fhaOd2XWqUQAYdWYrTROJ6Wm4/XjTfv0drJv8dBH5izHmt805sDGmRkRuAt7BCo992hjzuYhMt7c/CbyFFRq7FSs8dlJzjhX95BJrKnFJ+pXwCKpROOjc/FuhOnF+iUfzKJSmiURQdAWGGmNKAUTkXqxaT2OAVUCzBAWAMeYtLGHg/96Tfs8Nlo8koTTHVOLtVdGccMZAH0Xyk75ig3/Uk7PCTwFvE4YGmePp8t1Hj/h9H4rSkEiu3L4EphpXA/2MMeVAZVxmlWSidWa3tIibpGFRwHqNwpk9qesXAcZRCXdJw2d6Uo1CaUwkGsWLWDkMr9uvLwUW2s7tjU1/LHWJxpkdiyJu6XmD8l+xO9CZHWB6cp4PJeH4+jipoFAaE1JQ2CU0FmCZh07H+neaboxZaQ+5Lq6zSxL1cfXhb2yxKeLmnBtorAhoDOTEG7F/0UIHajyJR3tmK00TUlAYY4yI/NUYMwzLH9E6iML0FIsibk5aaccK34o94EbsnPOsn4mzyownj8id2S3xxympSSSmp2UiMsIYsyLus3EI0ZhKYlLELQ1Xsl4hW3z4EHs9e+wFq3POs9705OfAdZAgSzh2P4pDFZvYfrjpnipLl3zArFl/oEeHTNasKuV3v5nLNeMnJGqWSpKIRFCMBaaJyDagDGvpYYwxg+I6syQSrSmipUXc0vH25L0R33rLjQwa2Z6TLuzImlWrOemi8UmemY2f6cmb7O9qxVFPYNU521L8FFuKQww7Gib/1qp8cOXBCqZePolBpwymoKAgAXNMH1JNK4tEUFwU91k4jOaYIlpSxC0dTR5VlVYP8SkzOuPKzaQEWPji81w4/GeOuDD8TU91dtRTOmp2kVJ26CqqaioZ0BPaZAVfuhw+fJiN61Zyat8adublUt25LWecl8MlV4zkoftVswhHnalmb+lnvP/+uzz2x0fo0TGLvcXV3HLT/Vw7LuFZAFERSQmPbQAi0p1W0AIV/CM/ErXWTz+doqosE1xQ1q2+yVPbKhLQqS0y/E1PJDgT34lUVwzi650ncs0pHejbLfhtwePx8KOz+3H2teWs7NWX4d/pxuX3fIvLgd/++CY6d+nKkCFDHPH7OpFVu+awdu8zcDTc8Gg/3/tlPM3Xu0bTv9fg5E0uDGGXUCJymYhswSoG+CFQCLwd53klFa9GEVjVNX74O3k9Hk9CjhlvRh19M3++azsDln7FORu+YuB7X7D0nUMJ6NQWIQGtWr2mxtZrehJfeGzTY7z+uMufa8OGN/fR82AJuRVWitVxo3K4fepVzcojai2UVFnVhw5sr6LXgcP0OnCY7GqrY8PXOz9L5tTCEomu/SvgVOALY0x/4Bzgk7jOKslEEx4bC95795++5+lyofXscTQ3T5zJ1fdWcvV9dVzxYC2zn0xEp7bIqNce/Gs9tV6NwhWBoADLH7dsxRo2bjL0fWsLg9Z+A8AFV3bmD09047mfZjL1hkls2rQpzjNOPbyJnf94xsPRb2/lO+u20nb/EQB6HNU9mVMLSySCotoYUwS4RMRljFkCDI7vtJJMAsuMezweHn6ovlngkhvKmTFtclpoFteMn8DmrduYs+g9Nm/d5igbdtDGSmnoK4qUSDQKLwUFBTw57xnGPpXDhN8bqipqqW6XzdajuvLZkL4c1a6Sb48cEnLB4/F4WLFiRVr8n0eK9/9s8g3TGftUDkNnd2DFdut/rkOHvGROLSyRXBmHRCQPWAq8ICIzscp4pC2JzCQuLCykXbl1dWZX1wQk7KUDbrebESNGOEaT8OFbBBhNuKNeUETajsK7CHj0iVf404+20WNJISVF1XTp1YZPf5XDM+MqmTxpIp980tj40NKSN6mKN2hi9OgxvgXUmDFjrW3UJnNqYYnkyliHVbn1VuAfwJfA5nhOKtkksix2fn4+W7ZXMuwv67ju08+albCnRE+wDnetWaNw+TeaihC3283555/PPT9/gu/fW8b2dVarmllb23Hti9C9XTXnnHk6U6bc4PuMf8mbVTOK00qDDod/3xPvAqpNmxxrm8OLMUaUR2Gss6gD/gwgIuvjOqskk8hGNl4H4XktSdhToqY+c9w4MnM80fhMT8347DXjJzDolME8+vLFACzfm8ED09uzaB2ckAcrN73MzKc7Mu7q7/JF4X85/eyOdDkxkx3AgKzyZpS8SU28PoqAtgL45/M4lyYFhYjcCMwAjmkgGNqT5s5sEuzMbmnCntIMfELBvyigRj01tyZgQUEBp58xlmrWceVP+gBwQ8CID3h76weQAZfc08sXNpldVsm2oq9ahQbtNT35/5/5921xMqE0ihexwmB/Ddzh936JMeZAXGeVZBLdMxtalrCnRE9ghzstMx6tjyIYAwYcy6b963yvXXvK+O/2WnKyoKwK+n/rGPr3/xZ79uxh86bPGTA0l6rcNsyeM7dV/O8Hs1SkvEZhjCnGannqnFCVBOHEsthKbAkwPTmxsVKCaalGYe0j8Pubec92nj/riK9Y5tjbd7L5y+dwH+vGM9DDG99cQR0VXHn15S2YeepQvyDxExS+KsvO1iha75URAkeWxVZiiq+jG173G7hacdRTpHkUoWi4sOrYxqqoDHb5/W710Xxut5s2WVbWfk1defMPmkL4EnkDGpVZz53eq7z1XhkhMAnuma0kg3ofRf1F2np/73rTU0saFwUKikOVbVi/y3oeLJovK8OK+Kmpq2jBMVMHn4/Cz5ntFRpebcOpRBL11ApR01O649/hDg2PxeXrmd38fTQ0Pf3i7nsZe/0vmozmy3R5BUUr0SiCCIpUMT2poAhCMpzZSmLxmZ5Mnf7e+OlXMTQ9XXDBRWzeem2T0XxeQVHdWgSFz/Tk76NIDdOTCoogqDO7NeD1URjfaq41axQtyaPw7aPB9yciIaP5smxB8fmmdWTk9077yKdgFQDqNVsVFCmHOrPTn/rf1vhpFK13YeA99T8vKSM780iz9tGlRxWdu9W//sObZVRXHmpyfLde0LEz/G19LfPf2UnXbpXk5kZW8ygrA/5ndDtO7td0Nz6n4TM9+d12vRqF+iiCICJdgEVAPlbZ8v8xxhwMMq4QKAFqgRpjzPBEzE+d2elPgOnJp0G23oS7o7tlsK6wmpJyQ3P1ipxOQme/14dKDRUVTa+UO9ZaUU9t2ueRW9mXijqoKIl8Zb1ia1VKCQpvPSd/jcKV6nkUceYO4F/GmIdF5A779c+bGDvWGLM/cVMDdWa3Buqd2bV1Vk+AA0UH6H1U51AfSlsuH9mOM05oS20LMu42FrXlC7/l3q2XdaR9dsegY9evX8/bK/9JtzEZXFV7D/0rfsalz+Xx4KznGTQodJfldV9Xs+iTI9Q4exHeCK8wcKVgHkWyBMXlwFn28z8DH9C0oEg46txMf7yLgC+3buFwxiFy2mcwatgpPProPEeVQ08kXfJa9v/erjRQI+ual0nHtsG1tEHH9+HZt4o5ni60zTjE7p372PJVDoOO74O7Q2jNrnOedVNtiVBLBvW+MP9aT15ntrMFRbLuhD2MMbsB7L9Nde0wwLsiskpEpobaoYhMFZGVIrKypZUo1Zmd/nhNTwfNanJyrcvgje+1nkqm8aDh9RLK5+N2uzl77EUAPLYih7FP5URcDDPTvs+mmkZR53NmB6n15HDTU9wEhYi8JyIbgjyiydcfbYwZClwE/EhExjQ10Bgz1xgz3BgzvKXRE+rMTn/aZ1uF6/I6Z4JLyKmqZnCPurTqBZJoGl8voa+fQSdbLsdLr7k2aGOrppobZdi7rUk1jcJXPVZNTz6MMec2tU1E9opIT2PMbhHpCexrYh+77L/7RGQxMBKrgVJcUWd2+tO7w0jOPeoprhp3Lo9fVsmwdhVs2Km9QFpCQw0i3EIry9UWgH2ud3l9+3uwvX6bMYbaWuvmueprcG1z4XLV72/YiCwqD94GjIvN5BNAKNNTq9UowvAGMNF+PhF4veEAEckVkfbe58D5wIbETE9NT62B/r2GcMctj3PZw3V8e2ZuVOYPJRgN8yhCj+6RN4QsVy52SEHAA6kjI1N8D3EFjsnIqCCj7Yr4nUocCJpHYT+v0zyKoDwMvCwik4FvgO8CiEgv4CljzMVAD2CxvUrJBF40xvwjEZNLZIc7JbloL5DY0XhhFfr6cbcr4AenLIEGN8mVK1fxw++eS+HeMj7+Eb7qs6fPhv+sWs/hnO1sOHgPhsrYnkCcqQvWuEjzKJrGGFMEnBPk/V3Axfbzr4BTEjw1ax6+uvGqUbQGtBdIbIjW9ATem2ZglFP//GMo3FtF7zwY3BMw1t9eebByxWpOPLuffYDUEhTB8nVSpR+FLpmDoM5sRWkOjUt4NAe3280vH3iI7cUEVJ/dUQwjR44k29XGHpligiJIPwpXa3dmpzTqzFaUqGmYd9SShdZt//tTNm/exKmznqZPR0tITJl+EwUFBWzeu8Y+QFVLpptwgrVCTZU8ChUUQdA8CkWJnmjyKCJh7rz53HrbT1m+fDkjR46koKAAgExXtr3/5AoKj8cTlW/LVz02mI9CndmphzqzFaU5ROfMjoSCggKfgPDSJrOtfbjkmZ4WLVzIjOmTye+WTeH+Kp6YMz9sRn+wqCef6Uk1itRDndmKEj2xND2FIjujjX285GgUHo+HGdMns+SGcgb1Krf6gU+bzNnnnNukZmH1Ztc8irTC+Kpn6tejKJHS2PQUn+NkZ9qCwpUcjaKwsJD8btmB/cDDZPTXm5YkYAGaKnkUeicMRhAVUVGU0ERbwqO5eE1PLldyNIr8/HwK91eF7AfeEJ/ZqVFkmOZRpCzqzFaU6GlOHkVzqDc9VWKMSbiJ2O1288Sc+YydNrnJfuAN8ZqdXBJ4y9UOdymM5lEoSnOITR5FODJcWRjjwuWqpbq2huzMrKDjoo1KioZoM/qDObIhdZzZeicMQv2PqhqFokRKIx9FnG4vIkJdnRUiW1kT3Py0aOFCBg7ox/Tx53Hct/ry0IMPxLx8vNvtZsSIEREJofocioYO/9TIo1BBEQR1ZitK9DQyPcVxoWWMJSiqaioabfOPSrr928VIbQXP/PEeBh7Tj0UvLYzbnEJRb3oKLFeieRSpjDqzFaUZJMaZDWDqLD9FZW3jyCdvVFLPDuWMfRI+uNFbWLA8bBhr3OYbpGmR9VpNTymLOrMVJXoSlUcBoTUKb1TSP7+A/M5EFcYaL+pzKBr4KFIkj0I1iiCoM1tRoifWJTxCYguKippiqmpLAzZ17JLD7HlP8JObf0RVVQWr98FJPWHDbthTXkvvvu5Gn2kOLskk026+FA5vG9TGpieNekpZ1JmtKNETuLCK77VjjGV6+mjnDXy0M8iAAfCrtwcCsMp+ANx/Dfx95zgI9pkoETI4K/+XDOhyYQTzbVw51rsPcL4zWwVFENSZrSjNwS/jOM7XTkXpWWRkbadNlsEVZkFnjKGurg6Xy9Vo8VddXU1FRTkuEeqMoW3bHLKygofb+lNnqqk1VewuWRmZoAhSvgNUo0ht1JmtKFETWJoivhpFZcn/8N8vx3Hqcdnk5TbvOi0/coSn5z7JhEE1uPPAUwrPrc9kytTp5LRrF/KzVa5/UZZ1H1/uO8jeXUfCHqtWyiAbyiqEV/5dP75aqiEb9hVXB7zfEs4/pS0d2sX23qWCIgjqzFaU6PHXIuKtUeS1dQG1LPuiJWU8XBx/5gxWe192huPPhKX/BWjsJPenQ8c2DCyA/aXFfLIp9FiAtm3LGTQYyiqFZevqx7dvX0PBiXCwtJpPN4bfTySMHtiGDqHlXNSooAiCOrMVpRkEaBHxXWRNOKMda76qos6EH9sUZWVlPPrbh5k6spqjOsCewzB3eRb/e/sd5ObmhvxsFd3YB3TrWMaJp+aEPVY12ewF2udkcpXf+Era4QG6diDg/ZbQPif2370KiiCoM1tRoidAo4iz2bZ7xwwuGNLSG2sOtTuHMaNBzaarz+gW9pOFuzP4527IyijmolPCz6PoSCavbYYOOVlcVFA/fm9ZO974L3RsZ7hoYGwERTzQJXNQ1JmtKNEiCXRmN4XH42HFihURl+u4ZvwENm/dxpxF77F567awzYfAKg9y+ojRABQd2sW8uXPCfsZbRtzVKI8iNZzZeicMQlMlgRVFCUECndnB8K/vNHBA5OU6oqnZ5C0P8uMhZQDktM/gJz+e7hMWTQmq+vDYpjKzmycoohWMzUXvhEHQDneKEj2JdGY3xL++06oZxSy5oZwZ0ybH/AZaWFhI786Z/OZ9yKiqAeDD/83kjttvYd7cP3HC8f2Ycd15DDy2Lw899Es+3/gZy1cso+igNY+m8ijClfAIJhCaKxibQ1J8FCLyXeA+oAAYaYxZ2cS4C4GZQAbwlDHm4UTMT53ZihI9iUy4a0h917lyILBcRyzrOuXn57NtfxX9O0G72lpKyGTNuEE8PA7gKR56z7+/9+t8WvG6dffaZ73TuHFRfYe7hmXRva/Xrl7NHbff6uvPPevp38Gx/2DfUd/wyD+OJcMFtXWwu+JhFqydwxUD/0yntv1ids6QPGf2BuAqoEnjnlg62mzgPGAHsEJE3jDGbIz35HxqoGoUihIxicyjaIh/1zmrAGD4rnPNwe1288jvZ3LLTdPJ++ogZQXdMQg19i0j0wU1dfhu3pl+cqGqxtC147CA/Xk1iuKSfVz985Po3iGTfYdrOPOss/nwg/fp3jmTbidmM3tRH3KzoawKvqmbQ+eaLHI7ZlID1Nj7ym0D1XUlEAd/R1IEhTFmE4T9ZxoJbDXGfGWPfQm4HIi7oEA1CkWJmmSanprTda65TJk6DYDvz7iF/G4lfHOghod/9wfu+NmtzLqknEc/hCfHwfRXYdWt9Z8bOrsDcxaNhGPq38t0WaVITFYp37u/j99RvuB7Y+tfl9gPgM6A1LXhdxM38uJlpZx4FHy+B654PodlK9bSoU3fmJ+zk8NjewPb/V7vAEY1NVhEpgJTAfr2bdkXpc5sRWkOicujCEa0XedawpSp07jiyqsCjtWhQwemT/khVVUVlFVB4UHCajh52T3pUfcdPl2+iHO/VQ3AgSOwYjtccDxU1sDLH1dzQ9diBnQ2fFkEP/lbNq+9/BHt71nH+QGCcS69jzomyGxbTtwEhYi8BxwVZNNdxpjXI9lFkPeaTK8xxswF5gIMHz68BWk46sxWlOYQaHpKziLL7XYnrNdEw2N5BdW8uXMY95uH6JBrOHVWBcf0yGHXYYJqOCLCaf1uZtI5j3DtDeUM6gUfbIXZ8+H7N1tC5vMNcMG7cHzv9nxzoIYn5jxO354n0Hf8CQkTjHETFMaYc1u4ix3A0X6v+wC7WrjPiFBntqJETzJNT07B7XZz5113M2XqNAoLC8nLy6O0tDTkjTyY2WzK9MmMfWq+7/XMx//A4CFDG+0nUYLRyaanFcCxItIfqyjweODaRBxYndmK0gySnEfhJKK9gQczm919z/8lRFuIhGSFx14JzALcwN9FZK0x5gIR6YUVBnuxMaZGRG4C3sEKMHvaGPN5YmaoGoWiREvg9aLXTrQ0FC6JNKOFI1lRT4uBxUHe3wVc7Pf6LeCtBE7NOi7qzFaUaAk0PbVujSLdcLLpKeF4E1yq2lkN21u7+qwoUaGmp7RFl8w2/unwy5f/G1CNQlGiIVCL0GsnnVCNAkuTuO/hG5n1Sl9ysuBATlvqgEPFh+nZPtmzU5TUQE1P6YsKCqw6Mf16tKGsay5l9nt1tYYDu6usoFxFUcLihDwKJT6ooMCqE7N2XSn/9/ZmjnXDFg9cOz+DFf8ZmuypKUrKoHkU6YsKCqwwtJkzn+LSgHT4OY4JTVOUlEDU9JSuqKCwSWSdGEVJRwJEg5qe0goVFH44KcFFUVINdWanLyr2FUWJEX6CQjWKtEJ/TUVRYkJgkp1qFOmECgpFUWKCmp7SFxUUiqLECM2jSFf011QUJSYEJNzprSWt0F9TUZSYECgc1PSUTqigUBQlJmgJj/RFf01FUWKEOrPTFRUUiqLEBNE8irRFf01FUWJCYK8i1SjSCRUUiqLECDU9pSsqKBRFiQlqekpf9NdUFCUmaB5F+qK/pqIoMUHzKNKXpAgKEfmuiHwuInUiMjzEuEIR+UxE1orIykTOUVGU6NA8ivQlWf0oNgBXAXMiGDvWGLM/zvNRFKXF+JueVKNIJ5IiKIwxm6BhWWJFUVIZ7Zmdvjj91zTAuyKySkSmhhooIlNFZKWIrPR4PAmanqIoXtT0lL7ETaMQkfeAo4JsussY83qEuxltjNklIt2Bf4rIZmPM0mADjTFzgbkAw4cPN82atKIoLUCFQ7oSN0FhjDk3BvvYZf/dJyKLgZFAUEGhKEpyUdNT+uLYX1NEckWkvfc5cD6WE1xRFAeipqf0JVnhsVeKyA7gNODvIvKO/X4vEXnLHtYD+FhE1gHLgb8bY/6RjPkqihIJ2jM7XUlW1NNiYHGQ93cBF9vPvwJOSfDUFEVpJlrCI33RX1NRlJjgb3qqrKhK4kyUWKOCQlGUmPDySy/7ni9+9RUWvbQwibNRYokKCkVRWozH4+FH02/wvb7ouBpmTJuM5jSlByooFEVpMYWFheR3y+a43UXkVlQxpKyYfl2zKCwsTPbUlBiQrFpPiqKkEfn5+RTur6Lz+9s4sxes3wXbinLIz89P9tSUGKCCQlGUFuN2u3liznzGTptMv65ZbCuq5ok583G73cmemhIDVFAoihITrhk/gbPPOdcyQ+Xnq5BII1RQKIoSM9xutwqINESd2YqiKEpIVFAoiqIoIVFBoSiKooREBYWiKIoSEhUUiqIoSkjEmPRrBiciHmBbMz/eDdgfw+mkAnrOrQM959ZBc8+5nzEmaMhaWgqKliAiK40xw5M9j0Si59w60HNuHcTjnNX0pCiKooREBYWiKIoSEhUUjZmb7AkkAT3n1oGec+sg5uesPgpFURQlJKpRKIqiKCFRQaEoiqKERAWFjYhcKCL/FZGtInJHsueTCETkaRHZJyIbkj2XRCAiR4vIEhHZJCKfi8gtyZ5TvBGRtiKyXETW2ed8f7LnlChEJENE1ojI35I9l0QgIoUi8pmIrBWRlTHdt/oorH8o4AvgPGAHsAKYYIzZmNSJxRkRGQOUAs8aY05K9nzijYj0BHoaY1aLSHtgFXBFOv/OIiJArjGmVESygI+BW4wxy5I8tbgjIrcBw4EOxphLkj2feCMihcBwY0zMEwxVo7AYCWw1xnxljKkCXgIuT/Kc4o4xZilwINnzSBTGmN3GmNX28xJgE9A7ubOKL8ai1H6ZZT/SfnUoIn2A7wBPJXsu6YAKCovewHa/1ztI8xtIa0dE8oEhwH+SPJW4Y5tg1gL7gH8aY9L+nIE/ArcDdUmeRyIxwLsiskpEpsZyxyooLCTIe2m/6mqtiEge8CrwE2PM4WTPJ94YY2qNMYOBPsBIEUlrM6OIXALsM8asSvZcEsxoY8xQ4CLgR7ZpOSaooLDYARzt97oPsCtJc1HiiG2nfxV4wRjzWrLnk0iMMYeAD4ALkzuTuDMauMy22b8EnC0izyd3SvHHGLPL/rsPWIxlUo8JKigsVgDHikh/EckGxgNvJHlOSoyxHbvzgU3GmN8nez6JQETcItLJfp4DnAtsTuqk4owx5hfGmD7GmHysa/l9Y8z3kjytuCIiuXaABiKSC5wPxCyaUQUFYIypAW4C3sFycL5sjPk8ubOKPyKyEPg3cLyI7BCRycmeU5wZDXwfa4W51n5cnOxJxZmewBIRWY+1IPqnMaZVhIu2MnoAH4vIOmA58HdjzD9itXMNj1UURVFCohqFoiiKEhIVFIqiKEpIVFAoiqIoIVFBoSiKooREBYWiKIoSEhUUihJjROQ+EflpiO1XiMgJiZyTorQEFRSKkniuAFRQKCmD5lEoSgwQkbuAH2AVl/RglTAvBqYC2cBWrGS/wcDf7G3FwDjg7IbjjDFHEnsGitI0KigUpYWIyDBgATAKyARWA08CzxhjiuwxDwB7jTGzRGQB8DdjzCv2tq7BxiX8RBSlCTKTPQFFSQPOABZ7tQAR8dYJO8m+8XcC8rBKxAQj0nGKkhTUR6EosSGYar4AuMkYczJwP9C2ic9GOk5RkoIKCkVpOUuBK0Ukx67gean9fntgt13a/Dq/8SX2NsKMUxRHoIJCUVqI3V51EbAWq9fFR/ame7A66P2TwNLeLwE/E5E1InJMiHGK4gjUma0oiqKERDUKRVEUJSQqKBRFUZSQqKBQFEVRQqKCQlEURQmJCgpFURQlJCooFEVRlJCooFAURVFC8v+bhqQdMjOWcQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Fit regression model\n",
    "regr_1 = DecisionTreeRegressor(max_depth=2)\n",
    "regr_2 = DecisionTreeRegressor(max_depth=5)\n",
    "regr_1.fit(X, y)\n",
    "regr_2.fit(X, y)\n",
    "\n",
    "# Predict\n",
    "X_test = np.arange(0.0, 5.0, 0.01)[:, np.newaxis]\n",
    "y_1 = regr_1.predict(X_test)\n",
    "y_2 = regr_2.predict(X_test)\n",
    "\n",
    "# Plot the results\n",
    "plt.figure()\n",
    "plt.scatter(X, y, s=20, edgecolor=\"black\", c=\"darkorange\", label=\"data\")\n",
    "plt.plot(X_test, y_1, color=\"cornflowerblue\", label=\"max_depth=2\", linewidth=2)\n",
    "plt.plot(X_test, y_2, color=\"yellowgreen\", label=\"max_depth=5\", linewidth=2)\n",
    "plt.xlabel(\"data\")\n",
    "plt.ylabel(\"target\")\n",
    "plt.title(\"Decision Tree Regression\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scikit-learn 的决策树参数"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "DecisionTreeClassifier(criterion=\"gini\",\n",
    "                 splitter=\"best\",\n",
    "                 max_depth=None,\n",
    "                 min_samples_split=2,\n",
    "                 min_samples_leaf=1,\n",
    "                 min_weight_fraction_leaf=0.,\n",
    "                 max_features=None,\n",
    "                 random_state=None,\n",
    "                 max_leaf_nodes=None,\n",
    "                 min_impurity_decrease=0.,\n",
    "                 min_impurity_split=None,\n",
    "                 class_weight=None,\n",
    "                 presort=False)\n",
    "\n",
    "参数含义：\n",
    "1.criterion:string, optional (default=\"gini\")\n",
    "            (1).criterion='gini',分裂节点时评价准则是Gini指数。\n",
    "            (2).criterion='entropy',分裂节点时的评价指标是信息增益。\n",
    "2.max_depth:int or None, optional (default=None)。指定树的最大深度。\n",
    "            如果为None，表示树的深度不限。直到所有的叶子节点都是纯净的，即叶子节点\n",
    "            中所有的样本点都属于同一个类别。或者每个叶子节点包含的样本数小于min_samples_split。\n",
    "3.splitter:string, optional (default=\"best\")。指定分裂节点时的策略。\n",
    "           (1).splitter='best',表示选择最优的分裂策略。\n",
    "           (2).splitter='random',表示选择最好的随机切分策略。\n",
    "4.min_samples_split:int, float, optional (default=2)。表示分裂一个内部节点需要的做少样本数。\n",
    "           (1).如果为整数，则min_samples_split就是最少样本数。\n",
    "           (2).如果为浮点数(0到1之间)，则每次分裂最少样本数为ceil(min_samples_split * n_samples)\n",
    "5.min_samples_leaf: int, float, optional (default=1)。指定每个叶子节点需要的最少样本数。\n",
    "           (1).如果为整数，则min_samples_split就是最少样本数。\n",
    "           (2).如果为浮点数(0到1之间)，则每个叶子节点最少样本数为ceil(min_samples_leaf * n_samples)\n",
    "6.min_weight_fraction_leaf:float, optional (default=0.)\n",
    "           指定叶子节点中样本的最小权重。\n",
    "7.max_features:int, float, string or None, optional (default=None).\n",
    "           搜寻最佳划分的时候考虑的特征数量。\n",
    "           (1).如果为整数，每次分裂只考虑max_features个特征。\n",
    "           (2).如果为浮点数(0到1之间)，每次切分只考虑int(max_features * n_features)个特征。\n",
    "           (3).如果为'auto'或者'sqrt',则每次切分只考虑sqrt(n_features)个特征\n",
    "           (4).如果为'log2',则每次切分只考虑log2(n_features)个特征。\n",
    "           (5).如果为None,则每次切分考虑n_features个特征。\n",
    "           (6).如果已经考虑了max_features个特征，但还是没有找到一个有效的切分，那么还会继续寻找\n",
    "           下一个特征，直到找到一个有效的切分为止。\n",
    "8.random_state:int, RandomState instance or None, optional (default=None)\n",
    "           (1).如果为整数，则它指定了随机数生成器的种子。\n",
    "           (2).如果为RandomState实例，则指定了随机数生成器。\n",
    "           (3).如果为None，则使用默认的随机数生成器。\n",
    "9.max_leaf_nodes: int or None, optional (default=None)。指定了叶子节点的最大数量。\n",
    "           (1).如果为None,叶子节点数量不限。\n",
    "           (2).如果为整数，则max_depth被忽略。\n",
    "10.min_impurity_decrease:float, optional (default=0.)\n",
    "         如果节点的分裂导致不纯度的减少(分裂后样本比分裂前更加纯净)大于或等于min_impurity_decrease，则分裂该节点。\n",
    "         加权不纯度的减少量计算公式为：\n",
    "         min_impurity_decrease=N_t / N * (impurity - N_t_R / N_t * right_impurity\n",
    "                            - N_t_L / N_t * left_impurity)\n",
    "         其中N是样本的总数，N_t是当前节点的样本数，N_t_L是分裂后左子节点的样本数，\n",
    "         N_t_R是分裂后右子节点的样本数。impurity指当前节点的基尼指数，right_impurity指\n",
    "         分裂后右子节点的基尼指数。left_impurity指分裂后左子节点的基尼指数。\n",
    "11.min_impurity_split:float\n",
    "         树生长过程中早停止的阈值。如果当前节点的不纯度高于阈值，节点将分裂，否则它是叶子节点。\n",
    "         这个参数已经被弃用。用min_impurity_decrease代替了min_impurity_split。\n",
    "12.class_weight:dict, list of dicts, \"balanced\" or None, default=None\n",
    "         类别权重的形式为{class_label: weight}\n",
    "         (1).如果没有给出每个类别的权重，则每个类别的权重都为1。\n",
    "         (2).如果class_weight='balanced'，则分类的权重与样本中每个类别出现的频率成反比。\n",
    "         计算公式为：n_samples / (n_classes * np.bincount(y))\n",
    "         (3).如果sample_weight提供了样本权重(由fit方法提供)，则这些权重都会乘以sample_weight。\n",
    "13.presort:bool, optional (default=False)\n",
    "        指定是否需要提前排序数据从而加速训练中寻找最优切分的过程。设置为True时，对于大数据集\n",
    "        会减慢总体的训练过程；但是对于一个小数据集或者设定了最大深度的情况下，会加速训练过程。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 决策树调参"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入库\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn import datasets\n",
    "from sklearn.model_selection import train_test_split\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(150, 4) (150,)\n"
     ]
    }
   ],
   "source": [
    "# 导入数据集\n",
    "X = datasets.load_iris()  # 以全部字典形式返回,有data,target,target_names三个键\n",
    "data = X.data\n",
    "target = X.target\n",
    "name = X.target_names\n",
    "x, y = datasets.load_iris(return_X_y=True)  # 能一次性取前2个\n",
    "print(x.shape, y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据分为训练集和测试集\n",
    "x_train, x_test, y_train, y_test = train_test_split(x,\n",
    "                                                    y,\n",
    "                                                    test_size=0.2,\n",
    "                                                    random_state=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最优分类器: {'criterion': 'gini', 'max_depth': 30, 'min_impurity_decrease': 0.2, 'min_samples_leaf': 3} 最优分数: 0.9416666666666665\n"
     ]
    }
   ],
   "source": [
    "# 用GridSearchCV寻找最优参数（字典）\n",
    "param = {\n",
    "    'criterion': ['gini'],\n",
    "    'max_depth': [30, 50, 60, 100],\n",
    "    'min_samples_leaf': [2, 3, 5, 10],\n",
    "    'min_impurity_decrease': [0.1, 0.2, 0.5]\n",
    "}\n",
    "grid = GridSearchCV(DecisionTreeClassifier(), param_grid=param, cv=6)\n",
    "grid.fit(x_train, y_train)\n",
    "print('最优分类器:', grid.best_params_, '最优分数:', grid.best_score_)  # 得到最优的参数和分值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 参考\n",
    "\n",
    "- https://github.com/fengdu78/lihang-code\n",
    "\n",
    "- 李航. 统计学习方法[M]. 北京: 清华大学出版社,2019.\n",
    "\n",
    "- https://scikit-learn.org"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
